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GLOSSARY 



PACE - Projects to Advance Creativity in Education. 

PATTERN - Planning Assistance Through Technical Evaluation of 
Relevance Numbers. 

MSL - Mobile Science Laboratory 

ESCP - ESCP is the Earth Science Curriculum Project. 

IPS - IPS is Introductory Physical Science. 

PIA - Refers to planning, implementation and analysis which were 
the three major sections of each project. 

National Objectives - The National Objectives refers to the set of 
overall national educational objectives. 

Approach Level - The approach level denotes in broad concept the 
methods of accomplishing the national educational objectives. 

Area Level - The area level states the various regions of edu- 
cational responsibilities in which the approaches to fulfilling 
national objectives could be used. 

Program Level - The program level is composed of a selection of 
PACE funded science oriented programs. 

Function Level - The function level is composed of the present and 
possible Mobile Science Laboratory operational areas. 

Curriculum Level - The curriculum level depicts an array of areas 
of science available for study in the Mobile Science Laboratory. 
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Categories Level - The categories level divided projects in groups 
of size amenable to relevance assignment. 

Project Level - The project level consists of all the student 
projects selected for the evaluation. 

Means Level - The first level of a project write-up consisting 
of planning, implementation and analysis. 

Elementary Program - The program that uses the Mobile Science 
Laboratory as a supplement to the elementary science curriculum. 

It provides laboratory facilities where none have been available. 

Secondary Program - The Mobile Science Laboratory program utilized 
by the junior and senior high school students during summer recess. 
The program consists of field trips for extended periods of time 
throughout Minnesota and Southern Iowa where the students perform 
various research and study programs in the field of science. 

Basic - The Basic program is for those students participating in 
the Secondary program for the first time. 

Phase I , II , and III - Phase I, II, and III denote participation 
of students in the Secondary Program for the second, third, and 
fourth times respectively. 

Relevance Network - The detailed structuring of the total process 
into subsets that contain decision nodes where relevance numbers 
are assigned. 

Relevance Numbers - The value of performing the function described 
at the decision node as determined by the experts. 
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Relevance Network Linkages - The relevance network linkages are 
the selected characteristics of the network options which relate 
them to criteria measures of the node of interest. 



Criteria Measures - The criteria measures state the dimensions of 
the criteria relevant to the node of interest. 



Relevance Guide Book - The relevance guide book is a document 
containing definitions and descriptions of all options and enter 
in the network. It explains the option linkages and criteria 
measures and establishes a set of common data on which to base 
relevance assignment. 
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Node - A node is a network section consisting of a decision point 
and its options. Each option becomes a decision point for those 

options under it. 



Evaluation Node - The evaluation node is that option in the rel- 
evance decision network which is being considered for the relevance 
assignments at any one time. 



Node Relevance - Node relevance is the sum of the products of each 
criterion weight multiplied by the assigned relevance value under 
it in an option row. Each option has a node relevance for each 

relevance assigner. 

Average Node Relevance - Average node relevance is the sum of 
the node relevance for each assigner divided by the number of 

balloters . 



Branch Relevance - Branch relevance is defined as the product of 
the average node relevances of all connecting nodes from the 
node of interest to national objectives. 
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Percent Standard Deviation - Percent Standard Deviation equals the 
Standard Deviation of relevance assignment at a node divided by 
the average node relevance multiplied by one hundred. 

Mean decile - Mean decile is defined as the average tenth into 
which a sample falls. 

Cross -Correlation - Cross -Correlation is defined as the relation 
between one sample and another sample. 

Transferable Learning - Transferable learning is knowledge learned 
in one area which enables better problem solving in another area. 

Educator Balloting Session - A discussion and relevance number 
assigning meeting of the educators. 

Student Balloting Session - A discussion and relevance assigning 
meeting of the students. 




I. INTRODUCTION 



The purpose of tliis contract was to evaluate the educational 
merit of the Mobile Science Laboratory, a project funded under 
a tliree-year PACE grant. The methodology used for evaluation has 
been employed in industry and medicine for planning and decision 
making and is known as PATTERN (an acronym for Planning Assistance 
Through Technical Evaluation of Relevance Numbers). Evaluation 
of a PACE program particularly must of necessity be a decision 

process . 

Under this contract, ALJ Associates, Inc. constructed a 
relevance network and evaluation criteria, conducted relevance 
assignment sessions with Albert Lea students and teachers. To 
these ends, ALJ Associates, Inc. representatives met frequently 
with the Program Director and the committee of educators. The 
atmosphere of objectivity that prevailed greatly enhanced the 
effectiveness of the evaluation. 

This final report is intended both as a report of contract 
activity and as a thorough examination of the Mobile Science 
program. 

This report contains the following sections: 

Section I. Introduction 

Section II. Results, a capsule presentation of the more 
significant results of the MSL evaluation. 

Section III. Recommendations, considerations for program 

improvement which should be implemented. 

Section IV. Contract Summary, a task-by-task summary of 

work on the contract. 

Section V. MSL Description, a discussion of the Mobile 
Science Laboratory considering its genesis, its composition and 
its funding. 
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Section VI. Study Procedure, a presentation of PATTERN as 
it was applied to tliis evaluation. 

Section VII. Analysis , on analysis of relevance results. 

Section VIII. Appendices, contained study personnel 
lesumes and comments, relevance guide book, computer flow 
diagram, and sample of student project cards and report. 
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II. RESULTS 



The section states some of the more significant results 
which have developed in this evaluation of the Mobile Science 

Laboratory. For a detailed analysis that supports the results, 
see Section VIII. 

A. General Results 

The Mobile Science Laboratory secondary program clearly 
induces behavioral changes in its student participants. It has 
shown that students who had more exposure to the MSL program 
were better able to analyze and implement in their science 
projects than students in the same grade who had not had as 
much experience in the program. 

The Mobile Science Laboratory elementary program has been 
a great motivational tool for use by elementary teachers in 
creating interest and excitement about science. The greater 
confidence and interest by elementary girls in science has 
been a direct result of the program. The general level of 
science education has been on the increase in District 241 and 
has become particularly noticeable in the last three years. 

The MSL program at the elementary level has been attributed 
as a substantial causal factor in the increase. 



B. Specific Results 



MSL Elementary Program 



u 

u 

u 



was the most beneficial application of the MSL 
provided resource teachers to guide elementary teachers 
if would have wide application in elementary in-service training 
if was a great motivational device 



ERIC 



II-2 



has created interest in learning which has carried over 
to other subjects 

^ has been as popular in parochial schools as in public schools 
lias enhanced the scientific base of the students of District 
241 

^ was the only science teaching support provided to elementary 
teachers 

^ increased regular teaching of science from 30^ to 85v by 
elementary teachers 

^had low student/teacher ratio of 15 to 1 

MSL Secondary Program 

preached 200 secondary students in the summertime 
^provided the student an opportunity to design his own project 
fallowed extensive use of the discovery method of teaching 
^changed the outlooks of both teachers and students 
'A'would provide in-service training in techniques of working 
with children and other people 

^gave students and teachers new feeling for each others points 
of view through living together 
-^had low student/teacher ratio of 15 to 1 
"^would provide a new kind of adult education opportunity 
^provided to students in working out an orderly, well-thought 
through approach to a project 
^demonstrated that progressive and immediate improvement 
occurs for student participants 
^demonstrated greater educational benefits to those students 
in the program for the longest times 
^showed that substantial improvement could be made in student 
analytic ability with participation in the MSL 
^showed that better balanced projects wore done by students 
who had participated in the program longest 
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III. RECOMMENDATIONS 

There is no question that the MSL Program has the critical 
far-reaching impact on District 241 needs for developing the 
science capability that is so necessary in preparing students 
and teachers, as well as the community for fruitful achievement 
in today’s competitive environment of exploding technology and 
rapid scientific advance. 

The following recommendations for program improvement should 
be seriously considered for implementation as the presently con- 
stituted program continues with its obvious high priority. It 
is imperative that the Program Director be relieved of some of 
his full-time teaching load to continue his excellent adminis- 
tration and management of the MSL program and further, that 
he be charged with the responsibility to implement the program 
recommended below. 

A special Science Advisory Committee for District 241 
should be immediately established to integrate the overall 
science program from the elementary through the secondary level 
to the needs of the entire community. This committee should 
be composed of educators, parents, students and key community 
leaders. Its first task should be to study this evaluation 
in depth and to define a technical and financial program to 
extend the inherent innovative characteristics of the MSL for 
much broader community utility. 

The MSL should continue to be used as the broad-based 
behavioral investigative tool to further demonstrate the 
efficacy of this innovative evaluation technique and to enable 
cumulative use of experience obtained in this study. 



O Due to the District need for teachers trained in both 
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elementary science and secondary field science, a significant 
in-service teacher training program should be implemented with 
credit allowance made for participants towards their salary 
schedule . 



a teaching role but rather should concentrate on development of 
study units, providing teacher training and support as needed. 

A substantially greater effort should be made to increase 
the availability of MSL to students outside the district. 

O Credits should be given toward graduation for student 
participation in grades ten through twelve. 



siring enrollment in the summer program should depend on 
"ability to pay" and should in no case be the limiting factor 
on student selection. 




The elementary resource teachers should not continue in 




Tuition fees, if required for secondary students de- 
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IV. CONTRACT SUMMARY 

This final report covers the work performed under contract 
No. 1025 during the period from 13 September 1968 to 1 March 1969. 
The study tasks referenced are those submitted by ALJ Associates, 
Inc. in Proposal No. P7158 dated 15 July 1968. For each task, a 
statement of progress and the major supporting data prepared by 
ALJ Associates, Inc. are given. 

In general, the contract progressed on schedule. Some 
problems arose in air freight exchanges of critical time-phased 
data between ALJ Associates, Inc. and the Program Director. 

Every effort was made to ensure proper coordination of the 
program and communications with MSL personnel. In this regard 
29 man-days were spent in Albert Lea by ALJ Associates, Inc. 
personnel, where the original plan called for 22. An additional 
11 man-days were spent working with the Program Director and staff 
at the ALJ Associates, Inc. facilities. 

Task I. Structuring the Mobile Science Laboratory Problem. 

Using data collected in contract 1020 "Preliminary Evalu- 
ation Study", ALJ Associates, Inc. developed a complete relevance 
network for the Mobile Science Laboratory and appropriate cri- 
eria for levels where relevance was assigned. Several relevance 
structures were reviewed with the Program Director before selection 
of the final one was made. An eight- level network was developed 
ranging from national education objectives to PACE programs to 
student projects and their three parts. 

Task II. Critique 

ALJ Associates, Inc. met with the Program Director throughout 
the contract. Two meetings with a panel of educators were also 
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held for review of methodology. The panel consisted of educators 
from elementary school, junior and senior high school, and the 
Minnesota State Department of Education. (The resumes of the 
Evaluation Committee are in Appendix A.) The first session 
reviewed overall methodology and the second session reviewed 
questionnaires which were distributed to provide data for the 
Relevance Guide Books. 

The meetings with the Program Director were held alternately 
in Albert Lea and in Washington, D.C. They covered selection 
of the relevance network and criteria, review of questionnaires, 
and review of the final report. 

Task III. Student Evaluation 

All edited projects received from the Program Director were 
reviewed and categorized in groups. The projects were stored on 
50 MT/ST magnetic tapes that were delivered as part of the contract. 
ALJ Associates, Inc. coded the entire relevance network and provided 
computer generated ballots for student relevance assignment made 
27-28 December 1568. 

ALJ Associates, Inc. provided seven (7) copies of all 
projects to be used as the Relevance Guide Book for Project and 
Means Level relevance assignment. 

ALJ Associates, Inc. personnel monitored student assignment 
sessions which were led by Albert Lea educators and members of 
ALJ Associates, Inc. 

Task IV. Data Analysis 

The completed student relevance ballots were returned by 
ALJ Associates, Inc. where they were examined for errors and 
corrected. Ballot data including assigned relevance number. 



o 
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student name and organization, and project number where key- 
punched. The computer program used this data to calculate node 
relevance, average relevance and percent standard deviation. 

ALJ Associates, Inc. then analyzed and evaluated the 
results for students at the Project and Means Levels. 

Task V Education Evaluation 

A set of questionnaires was developed to obtain information 
for the Relevance Guide Book to be used at the Function and 
Curriculum Levels. ALJ Associates, Inc. and Albert Lea educators 
jointly prepared a series of questionnaires for administrators, 
counselors, educators, parents and students. ALJ Associates, Inc. 
processed the questionnaires and produced fifteen (15) copies 
of the "Function and Curriculum Level Guide Book." "The Program 
Level Relevance Guide Book" was prepared from the letters sent to 
Project Directors and fifteen (15) copies were produced for edu- 
cator use. Copies of the "Project and PIA Level Guide Book" 
used by students were also provided for the educator evaluation 
meeting held 16-19 January 1969. 

Educators assigned relevance in sessions conducted by ALJ 
Associates, Inc. personnel. Assignment sessions for the Function 
and Program Levels were recorded for use in analyzing results. 

Task VI. Data Analysis 



All data from educator relevance ballots were checked for 
errors and key-punched. The computer program used this data 
to calculate node relevance, average relevance and percent 
deviation. The tapes recorded at the relevance assignment 
sessions were transcribed and analyzed for commentary. 



IV-4 



Correlation and comparison of educator and student rankings 
and results were made as well as student and educator criteria 
rankings at all appropriate network nodes. 

Task VII. Report 

Twenty-Five (25) copies of this final report were submitted 
to the Program Director. An outline and draft of the report 
were reviewed and approved by the Program Director. 
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V. MOBILE SCIENCE LABORATORY (MSL) DESCRIPTION 
A. Project History 

The Mobile Science Laboratory program in Albert Lea 
developed from a summer science program conducted in 1965. 

The 1965 program was conducted as a pilot study to determine 
interest and utility of a secondary summer field science 
program. Tlie principal drawback of the program was that a 
$15.00 fee was required from each participant. The school 
district provided classroom space and equipment (plant 
presses, microscopes, etc.) and paid instruction costs. 

This experimental science program provided several half- 
day trips and one full-day trip each week of the session. 
Additional time was spent in the classroom laboratory identi- 
fying finds and planning further activities . Each of the 
thirty students was required to work on an individual and on 
a group project. 

With this experience completed and evaluated as a useful, 
supplemental addition to Albert Lea education, an extensive 
planning session was held to develop a program for which to 
obtain a PACE grant. A wide range of community people par- 
ticipated including parents, educators, principals and con- 
servation experts. 

The PACE grant was obtained and the first mobile laboratory 
constructed. The PACE program structured the activities into 
two phases: a Basic Summer Science Program and a Mobile Science 
Program. The Basic Program was similar to the pilot program 
consisting of three half-day trips with two full-day excursions. 
Students were returned home everyday and no tuition fee was 
charged. Tlie Mobile Program had the prerequisite of completing 
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the Basic Program. It included an extended stay in the field 
for its thirty students. 

Review of the first year of the program pointed up the 
need for another laboratory which was then built. The review 
also noted that full-time teaching responsibility of the 
Program Director and the MSL instructors during the regular 
school year had precluded either the use of the laboratory by 
the elementary schools or development of a coherent elementary 
program. Authorization for two teachers part-time to develop 
and to teach the elementary program was obtained, launching 
this program the second year. 

The summer program grew to 200 students by the third year 
and the elementary program contacted nearly 4,300 students 
during its second year of operation. 

B. Projea Funding 

The pilot program was supported by the school district and 
by tuition fees. The PACE grant, a three-year grant with 
yearly reviews, wholly supported the Mobile Science Laboratory. 
The first grant for 13 months (June 1966 - July 1967) was for 
$42,000. The second grant for 14 months (July 1967 - Sept. 1968) 
was for $115,000. Third grant for 9 months (Sept. 1968 - June 
1969) was for $89,000. 

C. Equipment 

The Mobile Science Laboratory equipment consists of two 
trailer- laboratories , a kitchen trailer, a significant library 
of Field Service reference books, films and pamphlets. The 
first MSL trailer is a 40' Fruehauf semi-trailer which was 
purchased used and then remodeled. A portable 5.0 KW generator, 
light fixtures and outlets were installed. A variety of 
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cabinets and work surfaces were built-in and the trailer floor 
was tiled. Water and heat were also installed. The trailer 
provided sixteen working spaces for students, an area for the 
instructor, a photography laboratory, and a book storage area 
for library. The second MSL laboratory did not include a photo 
lab and increased the number of student stations to twenty eight. 

In addition, other modifications were made to increase its oper- 
ational efficiency based on experience with the first lab. 

D. Evaluation Description 

While the MSL is a semi-trailer with certain fixed specif- 
ications, it is considered something rather different for evalu- 
ation purposes. It was interpreted as a device built to accomplish 
specific educational objectives. To this end, ALJ Associates, Inc. 
viewed the MSL as shown in Figure V-1. 

E. Applications 

The MSL was used in the elementary program as a supplemental 
science resource and experiment facility. During a summer work 
session, the Science Resource teacher from the MSL and an elemen- 
tary teacher from each grade met and planned a series of science 
experiments and lessons. The group also set forth objectives 
for the elementary program. The Resource teacher then developed 
study guide sheets to accompany each lesson. 

Before the MSL arrived at the elementary school, a planning 
conference was conducted for all teachers there. The objectives 
and a review of the science units were discussed as was the 
supplemental nature of the activity. A tentative use schedule 
was set up. 



When the MSL arrived at the school, the actual schedule was 
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formed. Classroom teachers were briefed again on units and MSL 
use began. 

In general, only half the class could attend at any given 
time. The other part of the class was monitored by a teaching 
aid until their turn. 

Units in the MSL included shadows, magnetism, rocks and 
minerals, sound, machines, light, weather, electricity, and 
chemical change. They varied in number of lessons according 
to grade level. Most were taught by the Resource teacher witli 
the elementary teacher in a supportive role. 

Some field trips were made with fifth and sixth graders 
using the MSL, but these were of necessity of short range and 
short duration. 

The MSL was used in the secondary program during the summer. 
Depending on the phase of the program, the duration of the field 
trip varied. The Basic Program provided three half-day trips, 
two full-day trips and one day in the classroom. Students were 
returned home after each day. Phase I provided for one three 
day field trip each week for four weeks with two half-day 
sessions a week at the school site. Phase I was run in double 
sessions in the third year, providing for six day usage of the 
MSL. Phase II and III provided for one three-week field trip 
with one week at the school site. 

Each field trip was preceded by a planning session where the 
student wrote down his ideas for the project and the way he would 
collect and analyze his data. The students were provided general 
guidance and advice on request, but there was no fixed plan for 
action, at least in the advanced stages of the project. Students 
collected what was necessary and performed their experiments 
or mounted their collections. There was the opportunity for those 
desiring it to write study papers on what they had done during tlic 

pro j ect . 
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In introducing his bill to establish a President's Ad- 
visory Staff on Scienctifc Information Management (PASSIM) in 
the Executive Office of the President (S.J. Res. 202) , the 
then Senator Humphrey clearly stated the need for numerical 
planning techniques in high government and industrial manage- 
ment offices and referred to PATTERN; 

"The human mind has difficulty in considering more than 
10 or 20 factors at the same time in making decisions. 

Yet decision making problems of the space age may require 
thousands or even hundreds of thousands of factors and 
sub-factors to be considered. During the past two decades 
of rapid change, the human mind has remained relatively 
static in its capability, while the complexity of de- 
cisionmaking at certain levels of Government and industry 
has increased a thousand fold or more. The solution, 
therefore, rests with developing new techniques which will 
permit the decisionmaker to successfully deal with problems 
involving thousands of factors, but limits the number of 
factors which must be simultaneously considered to the 
limited capacity of the human mind. 

" PATTERN is definitely a milestone. (It) utilized 

such diverse disciplines as history and political science, 
economics, mathematics, science, and engineering, to 
incorporate into the decis ion- aiding techniques of equal 
diversity". 



The above statement provides a simple explanation that be- 
comes the very basis of the entire philosopliy of PATTERN and 
is essential to the understanding of the methodology. The human 
mind is simply not capable in assimilating and correlating the 
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multifarious data required in making a timely and knowledgeable 
decision. The problem is demonstrated in Figure VI -1 wliich 
illustrates the manner in which the total quantity of pertinent 
information increases and why the PATTERN technique had to be 
developed. 

This is the basis of the relevance tree structure. The 
decision process involves the consideration of much more in- 
formation than the human mind can grasp at one time. As the 
thought processes of the education evaluation process proceeds 
from broad approaches, on to areas of study, programs and func- 
tions to perform, there is an exponential rise in the quantity 
of pertinent information. 

The PATTERN process allows one to continuously dissect the 
decision problem into workable elements, make the appropriate 
decisions with expertise at each node, and recombine tJie results 
with the computer to arrive at a meaningful decision knowing 
that all alternatives have been objectively analyzed. 

B. General Study Methodology 

PATTERN uses the principles of decision trees that have 
been discussed for years as one means of displaying information 
in a simple pay-off matrix. The PATTERN techniques liavc extended 
to the decision relevance network which offers a more lucid means 
of arraying information when the problem is very large. The 
evaluation/decision network is made up of a scries of nodes 
and branches. The node represents the decision that has to be 
made. The branches represent the alternate courses of action, 
options, for that decision. The criteria represents the \arious 
factors, pay-offs, objectives, etc. that the decision will be 
based on. Those direct variables that directly couple the 
required information to the criteria are called measures. Hacli 
branch, representing an option, will support or meet tlie retjuire- 
ments of the decision as reflected in it meeting the criteria. 
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THE INFORIMTION PROBLEM 




Figure VI-1. 
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Tlie direct intcrrclat ionsliips of tliose factors of each option 
that perform the supporting function to the measures are called 
linkages. (A discussion follows that will describe these terms 
in detail.) 

The wide variety of information that is reciuired to make 
the decision is contained in the Relevance Guide Book. Hence, 
the Relevance Guide Book must contain: (]) a detailed set of 
definitions as to the content and impact of each decisioii, (2) 
the supporting data to allow a complete understanding of the 
criteria, (5) measures to evaluate all facets of the pay-ofr 
function, and (4) information that allows the assessment o 1' 
the contribution of each of the linkages for each option to 
the criteria. 

1. Network Structuring Considerations 

Some of the common errors in network analysis will be 
discussed at this point to aid the reader in reviewing the 
detailed application to the MSL evaluation. It is emphasized 
that we are talking about a DECISION network. One of the most 
frequent pitfalls that one encounters in constructing a network 
is the subtle tendency to construct a CATEGORIZATION or 
taxonomy network. A simple categorization is much easier to 
construct and often appears in a first trial exercise. It is 
detected when one tries to develop meaningful criteria, wliich 
is almost impossible for a categorization network. (In this 
evaluation one level was purposely made a categorization level, 
but no attempt was made to develop criteria or assign relevance 
numbers . ) 

Another problem continuously encountered in network struc- 
turing is the tendency to want to make the number of branches 
equal at each node. This is normally not a subject for argument 
until the phase of relevance number assignment is approached. 
Then one becomes concerned that the option that is presented as 
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a set of two items will automatically receive higher relevance 
number than an option that is in a set of ten items. llie next 
tendency is to grab some normalizing factor to ’’solve" tlie 
"obvious" bias tliat has been introduced by not iiaving a network 
structure containing an array of equal branch nodes. Tin' iiornal- 
izing teclinique always proposed is to simpi> calculate tlie 
average re 1 evance , i.e., 1/n and mu It ip ly all tli e i e 1 e vmih c' 
numbers at each node by this factor for the particulai' node 
under consideration. It is emphasized that tliere is no -latlie- 
matical proof yet developed that can substantiate the bins or 
the selection of any normalized factor to compensate lor n 
situation that cannot be measured. All indications to date 
(including results of this MSL evaluation) show that tlie netuork 
structure can be built randomly as to number of branches, or 
levels which are simply means of portraying all tlie decisions 
and options. Add or subtract a branch and the definition ol 
the node has changed and will be reflected in a new relevance 
number because of different criteria and information lecjuiied 
to make the decision. 

Another important consideration is the factors associated 
with tree truncation. If one desires to constrain tlu' problc'r’. , 
either because of size, time, interest, availalile inlornation, 
ct al, (all of these necessitated a heavy truncation in this 
evaluation) it is often desirable to truncate a given node as 
lower levels of the tree are structured. This is perfectlv 
permissable as long as the particular branch to be continuevl 
has been placed into proper perspective at tlie node ol interest 
by having the other options completely defined so tliat one is 
completely sure of the information content of the continuing 
branch . 



A node is not considered to be completely structured until 
all the options have been determined and related to tlie d('c is ion 
and criteria measures through their linkages. It gc'tierally re 
quires that a less detailed network structure at all higher 
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levels be constructed before this assessment can be determined 
(This was done for the MSL evaluation.) There is no mathematical 
need to assign relevance numbers at the level above tlie continuing 
option branch (ie, even at the decision node where it starts) as 
the number will factor out of the calculations and simply result i 
a definition of the value of the constrained set (scaling) . As 
the truncated node is subsequently "filled out" or otlier nodes 
added, the network is simply expanded to reflect the larger set. 

However, it has been found through experience tliat later 
expansion of the tree and a more meaningful unders tanding of 
a truncated network is greatly enhanced if as much consideration 
as possible is afforded to the higher levels that will ultimately 
bear on the decisions. Hence, it is well worth the initial 
effort to place the complete problem into preliminary perspective 
by totally structuring the network at higher levels to the detail 
of definitions, measures for the criteria, and options before 
embarking on a truncated version. It is not required that a 
comprehensive Relevance Guide Book be prepared for the higher 
levels, but an identification of the information considered 
important helps in establishing the criteria and supporting 
data for the lower truncated branches. The MSL Evaluation 
network was carefully structured so that it may l:>e easily 
expanded for future work. 

2. The Decision Process 

The key variables inherent in all knowledgeable decisions 
are a complete understanding of what is to be decided upon, the 
basis on which the decision is to be made, the array of options 
to be considered, and collection of only the information relevant 
to the actual decision. The network structure is used to define 
the problem and to present the various options that arc to be 
considered. 
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The network structure has often been used to illuminate 
factors that have not been considered in the analysis of a 
complex problem and to interrelate the various alternatives. 

But the usefulness ends at this point unless one adds tlio 
other most important ingredient of the decision process -- cri - 
teria. All decisions require some implicit or explicit basis 
on which they are made. Identif-loation of the propi')‘ im’ai'upe- 
ment of effeotivenest, values has long heen the most difficult 
problem facing the educational evaluation community. 

In PATTERN this payoff or objective function is embodied 
in the use of criteria. The wliole meaning (dimensionality) 
of the relevance numbers is measured in terms of the criteria. 

The criteria are used to organize and explain the wliole range 
of payoff phenomena in a small number of general statements, 
to aid in testing the option relationships and to predict their 
growth value, to be able to appraise the soundness of the 
various factors tliat bear on the decision, and to enable one 
to appraise the soundness of the decision to improve the 
performance over time. Hence, the criteria provide the 
guidance as to the complete data collection process. One of 
the most difficult problems in any decision is the ove ralnindance 
of data available tliat appears to be of value to the decision 
under consideration. Most of these data really have only a 
superficial impact and much of it is incorrect because it was 
collected for entirely different purposes and contains under- 
lying assumptions and constraints that are difficult to identify 
and interpret. 

A direct analogy of the decision process as utilized in 
PATTERN can be made with the simple feedback example of a summing 
amplifier. 
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Figure VI -2 illustrates the situation. 
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It is obvious that the transfer function oi' this system is: 

System Dynamics Gain 1 
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i Option Relevance 



i^^ Option 



1 + (Criteria) (System Dynamics Gain) Criteria 



where the approximation is true if the forward loop gain of the 
system transfer function is large, no matter how complex the 
dynamics may be. It is not necessary to be concerned with tlie 
internal workings of the black box represented by tlie system if 
we can understand and control it via the feedback loop expressed 
in terms of criteria. 

However, it is very important tliat all facets of the cri- 
teria be included and understood as the decision is analyzed. 

To ensure tliat the criteria are more tlian general statements 
that could be interpreted many ways they must be specifically 
tailored to the decision being analyzed. This is done througli 
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the use of measures as shown in Figure VI -3. For cxnmpU', 
Economic considerations may he an important factor in the 
analysis at many levels of the tree. These factors ma)’ ranr,e 
from the GNP to the specific cost of the language laboratory 
for a school. The measures describe the hard factcM's that must 
be considered in the decision to determine the ultimate xnalut' 
of a given option. The measures must be directly coin) led to 
the decision as they are used to establisli tlie criteria weigjit 
and form the outline of one facet of Tlie Relevance Guide Hook. 

The linkages are the factors that measure the contribution 
of an option to the decision payoff (Criteria). The only im- 
portant factors about an option are those relevant to the 
measures. Hence, in considering the utility of a laboratory, 
for example, we assume a’priori that the equipment will work 
to the level of performance determined in the design trade-off 
analysis the Program Director makes of the various a])p 1 i cat i ons 
and their interrelationships. So what? The issue is, what are 
the unique characteristics about this equipment and its appli- 
cations vis-a-vis its alternatives that make it an improved 
contributor to the educational function it is to j^eiddM'in. 

These unique cliaracter s i tic capabilities are the linkages that 
contribute to the payoff as expressed by the measures. 

The specific details of the MSL network structure, criteria 
relevance guide book and number assignment process constitute 
the remainder of this section. It is to be emphasized that the 
study has shown that tlie principles of decision theor\' as 
embodied in PATTERN were easily applied to this evaluation. 
Figure VI-4 is a basic flow diagram of the process tliat follows. 
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C. Application to the Evaluation 

1. Introduction 

This section provides a detailed discussion of tlie aj)p J i cat i on 
of PATTliRN to the MSL evaluation. 

2. Network Structure 

The relevance network structure is an integral part of tlie 
PATTliRN approach to evaluation. It provides the means for making a 
complete array of the options available to evaluators assigning 
relevance. Tlie network designates the decision points and the 
options available at each point. A decision point (node) is 
not complete until every option of interest has lieen determined 
and related to the decision. The network allows examination of 
factors that generally have not been included in tlie analysis 
of a complex problem. 

The basic principle used in structuring the network is to 
define the area to be investigated as the universal space and 
then to uniquely subdivide the space into smaller parts until 
the data can be evaluated in manageable pieces. 

To tliis end, several possible networks were investigated. 

One of the first was that which is evidenced in the organization 
of the Office of Education. This structure was rejected on 
tlie grounds that it was not cogent to examining I’ACli projects. 
Another possibility was the structuring of the Elementary and 
Secondary Education Act. However, the development of meaningful 
criteria was impossible and the applicability of the titles of 
the law was in question. The selected structure is shown in 
Figure VI-5. The first level of the network was the Approach 
Level. This enumerated the three principal areas of activity 
in education. The Area Level denoted major education subject 
areas under each approach. The Program Level identified a 
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selection of PACE programs. (See Section VI. C-4b for the 
discussion of selection rationale.) The network was structured 
only under the science area, using the principles of truncation 
discussed in the previous section. The Function Level stated the 
four activities for which the Mobile Science Laboratory could 
have been used. The elementary and secondary uses were 
necessarily emphasized at this stage of program development. 

Below the Function Level, activities were performed in a 
detailed reportable way only in the secondary area. The 
Curriculum Level identified the four areas of secondary science. 
Ihe Category Level was inserted to classify the student projects 
into manageable groups. The Project Level was composed of a 
selection of student projects while the PIA or Means Level 
stated the three phases of each project. 
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3. Criteria 
a. Introduction 

A criterion is the standard by which a judgment can be 
formed. It is a key part of the PATTERN approach to evaluation 
and decision-making. The criteria form the basis on which to 
assign relevance. The criteria organize and explain the benefit 
accrued through accomplishing any one of the decision alterna- 
tives. T}ic relevance number indicates the magnitude of the 
benefit^ but the criterion indicates the dimensionality of the 
be nefi t . 

For each criterion, one can develop measures which relate 
the specific impacts of the option to it. While each criterion 
makes a general statement about its unique area of benefits, the 
measures state the factors which are important to the criterion 
as the evaluator is measuring it. Only those factors stated as 
measures are relevant to the evaluation. 

Each criterion merits a full discussion in the Relevance 
Guide Book and in the relevance assignment session. The discus- 
sion in both includes delineation and delimitation of criterion 
measures . 

ALJ Associates, Inc. defined criteria at each level of the 
network. These criteria were discussed first with the Program 
Director and then with the Evaluation Committee prior to assign- 
ing relevance. 

ALJ Associates, Inc. endeavored to establish uniquely defin 
able benefits at each level of the network. These refined defin 
itions were then presented to the relevance assignment session 
with the view that specific measures would be enunciated at that 

time . 
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The criteria at each level of the network are presented 
here with the measures determined by the committee. The com- 
plete set of factors involved in assigning relevance is there- 
fore presumably represented. 

b. PIA Level 

How well did the student develop his study approach in 
each of the three project phases in terms of: 

1. organization of thoughts and ideas 

a. continuation of the original theme of the project 

b. flexibility in planning to cope with variable 
situations 

c. effective use of orientation information 

d. statement of an analytic methodology 

e. clear statement of objectives 

f. statement of collection rationale 

2. collection and development of sufficient data 

a. plan to collect sufficient data 

b. actually collect sufficient data 

c. analyze sufficient data 

d. handle expected problems 

e. handle unexpected events 

3. completion of the three phases of the problem, accept- 
ing the student's definition of the problem 

a. completion of plan 

b. completion of implementation 

c. completion of analysis 

d. recognition of the need for further study 
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4. fulfillment of phase expectations 

a. closeness of approach to expected goals 

b. conclusions consistent with data collected 



5. effective use of the MSL program 

a. use of all resources available in the field 
(personnel, equipment, facilities, location, 
student interaction) in each phase 
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These criteria had the objective of evaluating student 
performance on his project based on his strengths and weaknesses 
in developing each project phase. Planning, implementation 
and analysis were measured one against the others in yielding 
the types of benefits defined by the criteria. 

c. Project Level 

How well does this project compare with other projects 
in terms of: 

1. the best use of time 

a. accomplish what was set out to do 

b. budgeted free time 

c. use of obvious short cuts 

d. level of accomplishment 

e. work constructively performed the full time 

2. flexibility in coping with situations beyond the 
student’s control 

a. amount of student planning to overcome the situation 

b. success in producing good project 

3. organization of the study approach 
a. technical quality 
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b. use of scientific method 

c. scientific quality 

d. master plan for all three phases 

4. production of meaningful results and conclusions 

a. completion of master plan 

b. relative to other projects, not on absolute scale 

5. educational value 

a. value to the student in assigner’s opinion 

b. competence 

These criteria had the objective of evaluating student 
performance on his project in terms of what other students were 
accomplishing. Each project was evaluated against the others 
with regard to the measures. For example, one evaluation of 
student development in using the laboratory was made possible by 
comparing the number of years participation in the laboratory 
program to relevance of the project. 

p 

d. Category Level 

No criteria were developed at this level since the catagcries 
were established only to place similarly-oriented projects to- 
gether for ease of comparison. It was felt that no class of 
projects should be valued more important than any other. For 
example, a project on plant ecology inherently has no more edu- 
cational value than one in geology. Since no evaluation and no 
assigment of relevance needed to be made, no criteria were defined. 

e. Curriculum Level 

An assessment of the comprehensive benefits accrued by the 
participant in the MSL curricula. 
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1. physical benefits 

a. exposure to natural environment 

b. participation in sports 

c. various other forms of physical activity 

2. cognitive benefits 

a. independent self -directed study 

b. extension of previous learning 

c. real-world application 

d. laboratory experience 

e. individual attention 

3. social benefits 

a. group relatedness 

b. contact with community officials 

c. camping experience 

d. leadership 

4. affective benefits 

a. self-reliance 

b. scientific awareness 

c. motivation 

These criteria had the objective of differentiating the 
kind of experience to which the student was exposed. Each 
basic class of project required use of different materials 
and exposure to different individuals. 

f. Function Level 

1. participants (students, teachers, parents) 

a. degree of impact on motivation 

b. increase curiosity and attitudes 

c. increase knowledge and skills 
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d. increase familiarity with study techniques 



2 . 



facilities 

a. increase 

b. increase 

c. increase 



space 

materials 

equipment 



3. fulfill District 241 science education needs 

a. MSL as actually used 

b. existence of the MSL as a device to fulfill needs 

c. present strengths and weaknesses 

d. the district needs solved by MSL 



4. provide increased opportunity for new teaching devices 

a. MSL potentiality of future applications 

b. function as methods and techniques medium 

c. new teaching method exposure 

d. the district needs to be solved by MSL 

These criteria had the objective of evaluating the principal 
functions of the MSL. Each function was evaluated against the 
others in terms of its impact on students, its provision of 
facilities, its fulfillment of district needs and its potenti- 
alities. The impact of participation in the program is one of 
the key items in evaluation for PACE. This will provide insight 
into behavioral changes which are one of the important factors 
PACE hopes to induce. Facilities and District 241 needs are 
directly related to the program influence in its host area. In- 
sight into the item of continuation of the program on a local 
basis will be provided by these two criteria. Increased oppor- 
tunity evaluates the potentialities of the program in District 
2‘«J with inferences for PACE. 




g. Program Level 



1. technical performance 

a. evidence of promised behavioral changes 

2. cost 

a. economic feasibility 

b. accurate cost estimation 

c. effective and efficient use of funds 

3. resource usage 

a. staff suitability (size, qualifications) 

b. community -wide participation 

4. innovation 

a. use of the proposed innovative techniques 

b. degree of experimentation 

These criteria had the objective of evaluating PACE 
programs in terms of the degree to which one program met its 

I 

own expectations against the degree to which the others met 
their expectations. It was difficult to determine the edu- 
cational utility of the programs reviewed. For example, each 
program proposal promised certain behavioral changes. The 
actual development of these changes is certainly important 
but was hard to assess from the evaluation reports available. 

The innovative features of the program are important in 
evaluating its success. A program which is totally innovative 
might be more difficult to assess in terms of behavioral changes. 
Cost and i..anagement are always important to use of the idea 
by other parts of the education system. A well-managed program 
certainly deserves recognition for that over one that was much 
more expensive than proposed. Finally use of community resources 
to develop the program is a significant development requirement 
in PACE community participation in the planning stages is to be 
highly commended. 
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4. Relevance Guide Book 
a. General 

The Relevance Guide Book is an integral part of the 
PATTERN approach to evaluation. It provides the required 
common point of reference to the evaluators assigning rel- 
evance. The guide contains information pertinent to making 
knowledgable assignments of values to criterion weights and to 
alternative relevances. A complete guide book contains a 
detailed discussion of each criterion definition and its impli- 
cations. It also describes options and the possible advantages 
and disadvantages inherent in each. The options are discussed 
with particular reference to the way in which they apply to 
each criterion. 

The guide book does not attempt to promote any particular 
point of view, but rather presents as objective a description 
as its expert writers can develop. Almost any question has 
more than one position and, ideally, every effort is made to 
reflect each. 

The guide book fills the need arising from the fact that 
few individuals are intimately acquainted with all the details 
of the options of the network in the wide range required, 
particularly in its upper part where broad choices are defined. 

The evaluator who has been selected on the basis of his 
expertise at this particular level of the network, is presented 
with a complete statement of the necessary decisions, the basis 
on which to make the decision, and as complete an information 
set as is obtainable. 
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b. Specific Description 

ALJ Associates, Inc. developed a guide book for each of 
the levels of the network that were assigned relevance. The 
guidebook at the PIA and Project levels consisted of the 
student projects selected for analysis. The projects were 
organized by general category, e.g., fish projects, insect 
projects, etc., and bound together. For the PIA level the 
relevance assigners read each individual project and made 
their assessment of it. For the project level the projects 
were compared to each other. A printed sheet stating the 
criteria and measures accompained the books distributed to 

each group. 

The guide book at the Program level was developed after 
examination of over 200 proposals to the Office of Education. 

ALJ Associates, Inc. also used the responses obtained from our 
letters to the Program Directors of on-going PACE programs. 

Many of these letters evoked responses in the form of evaluations 
of the programs. From these data we selected nine representative 
programs which had sufficient data to allow inclusion in the 
network. Portions of the reports made available to us were 
selected as they applied to the criteria. The innovative 
features of the program and the program’s evaluation of its 
performance were the only criteria for which information could 
be collected. These were included in the form of excerpts from 
the reports. Cost data on the programs was sketchy as was the 
description of use of community resources. These data should be 
collected tlirough on-site investigation. 

Criteria were listed in the front of the book along with the 
key measure on whicli they were to be evaluated. 



( 




ALJ Associates, Inc. developed the guide at the Function and 
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Curriculum levels from the responses to a set of questionnaires 
(samples included in Appendix B-2) which were distributed to various 
groups in Albert Lea by the Program Director. The groups that he 
selected were the school administrators, counselors, elementary 
and secondary teachers, secondary students, and parents of 
elementary and secondary students. All administrators, counselors, 
and teachers in the system received the questionnaires. All 
secondary students who participated in the program recieved 
questionnaires. A random sample of parents of secondary and 
elementary students also received questionnaires. 

The questionnaires arose from the merging of a suggested set 
of questions from the educators' evaluation committee and from ALJ 
Associates, Inc. The merged set was then reviewed by the committee 
before it was distributed. Some delay occurred in the arrival of 
the questionnaires in Albert Lea because of air freight problems. 

The responses were to be received in Washington before the end of 
the Christmas holidays. Instead, ALJ Associates, Inc. received 
the questionnaires one week late, but their analyses and com- 
pilation into the guide book occurred in time for the assignment 
session in Albert Lea. 

The questionnaires were designed so that each response was 
directed at describing a particular face t of a criterion. They 
were not the normal type of survey questions, although some of 
these kinds of questions were included to provide information to 
the Program Director. 

The options were clearly defined at both of these levels 
without any necessity to expand upon them in the guide book. 
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5. Relevance Number Assignment Process 

a. Introduction 

The group of experts in the field is necessarily one of 
the more important developmental segments of the operation of 
management of a PATTERN relevance number project. These people 
must be the best available people in the area, and must be so 
fully dedicated to the task at hand that they will allow nothing 
to come between them and the proper assignment of relevance 
within the framework of Guide Book information. 

The above statement fully characterizes the group of 
Educators that served the Mobile Science Laboratory PATTERN 
Evaluation. During the time that they were assigning relevance 
numbers they allowed nothing to interfere with their being 
present. They devoted many of their nights during this time to 
reading the background information and preparing for the group 
presentation of their numbers and for the presentation of their 
views on the Guide Book data the next day. No group could have 
possibly given more of themselves and their thoughts than this 
group of devoted Educators. 

During the actual assignment sessions the amount of competi- 
tion between participants in the Educator section was felt to be 
one of the greatest, perhaps due to the fact that these were not 
professional management experts, these were people who were expert 
in the field of education. They had experience in interpersonal 
relationships and decision making at the personal interaction 
level rather than the corporate level. This, then, caused them 
to be able to sense the attitudes of others more readily, and to 
respond immediately as they would do in their relationships in 
their own classrooms. In addition to this, of course, they had 
performed as the monitor in the student balloting sessions and 




therefore had an advantage that many other balloteers had not 
received prior to their acting in the participant role. 

Additional momentum was received during the educator 
balloting from the fact that the balloteers were persons who 
had positions of responsibility in the MSL Program and in their 
regular assignments in the educational community. Such interest 
was aroused by this exposure to PATTERN that many of the Educators 
wished to take this useful tool with them and use its concepts 
in their positions in education. 
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b. Ballot Preparation 

Where the relevance network design is approaching the final 
stage, a deck of holerith cards are punched reflecting the net- 
work in code numbers. In each of this same deck of cards is 
punched the name of an element and the identification numbers 
of those criteria that are applicable to this opt^ion and this 
node. Another deck of cards is prepared containing the names 
of all criteria and an identifying sequence number (mentioned 
above). The names of the criteria are stored in relation to 
these identification numbers in the computer being used, and 
ballot sheets are prepared for all nodes of the network by the 
computer. One by one, each node of the network is examined and 
one ballot is printed for each member of the elite group that 
will assign relevance numbers. 

The ballot sheet is an answer sheet with spaces for the 
writing of relevance numbers into a blank matrix. (See Figure 
VI -6) At the top of the ballot can be found the name and the 
number of the node that is being evaluated. Beneath this will 
be found a short title opposite each of the numbers of the op- 
tions that have been assigned to be evaluated on this ballot. 
Along the top of the matrix are the numbers assigned to the 
criteria. Along the left hand side are the option numbers. At 
the bottom of the page, under the last line of the matrix will 
be a short title for each criterion with its respective number. 

This arrangement of data has been found to be most amenable 
for relevance number assignments and this is the way the com- 
puter has been instructed to print each of the ballots. At any 
time up until the actual assigning of the relevance numbers, it 
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Figure VI - 6 Sample Ballot Matrix 



CRITERIA 51 TECHNICAL fERFORMANCE-RLANNED VS ACTUALLY CONSTITUTED 

52 EXEMPLARY AND INNOVATIVE SCIENTIFIC EDUCATIONAL ACTIVITY 

53 COST ESTIMATES VS ACTUAL EFFICIENT APPLICATION OF FUNDS 

54 USE OF AVAILABLE RESOURCES AND STAFF 
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is possible to add to or delete from the list of criteria and 
relevance network options without disrupting the flow of the 
operation. This form is especially designed to facilitate the 
discussion of information as it is arranged in the Relevance 
Guide Book. 

c. Balloting Sessions 

The "experts" in the field were formed into a balloting 
committee to consider those topics in a set of options at each 
node using data specified in their Guide Book. The importance 
of each of these options was established using the criteria 
explained in detail in the Guide Book. 

(1) General Discussion . All of the experts were called to- 
gether prior to the actual number assigning sessions and 
a short presentation was made of the underlying factors 

of each of the selected criteria. The main purpose of the 
general discussion is to lend uniformity to the group dis- 
cussions that followed, and to give internal consistency to 
the interpretation of the Relevance Guide Book information. 

(2) Group Discussion . After the general discussion was fin- 
ished the committee broke up into uniform sized groups that 
were previously defined in a consultation between the 
Program Director and the monitor of the session from ALJ 
Associates, Inc. These groups were composed of people who 
v>rerc uniformly informed, but who also offered a variety of 
experiences so as to assure a high level of tension- 
environment information exchange. This tension is directly 
reflected in the high quality of the relevance numbers that 
arc assigned by the group. Tension environment was built 
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question by question by this group of peers. The output 
of these discussions were definitive descriptions of each 
of the variables. 

(2) Assignment of Criterion Relevance Numbers . This portion 

of the session was given over to the assignment of numbers 
to each criterion that was discussed in both of the sec- 
tions above. Before he evaluated each option using the 
single successive criteria, (order of number assignment is 
unimportant) , he addressed himself to the relative impor- 
tance (weighing) of the criteria with respect to each other, 
ensuring their sum total is one. 

(4) Group Examination of Numbers . Once each committee member 
had individually inserted his data in his matrix, all the 
matrices were presented to the full committee, and those 
who had widely divergent opinions were asked to explain 

why they gave that particular weight number to the criterion. 
This usually involves much debate and transferring of ideas 
in a highly tense atmosphere. At the time that there 
seemed to be minimum of new information being presented 
in the discussion, the monitor called a halt and the group 
goes to the next section of the session. 

(5) Reassignment of Criterion Relevance Numbers . After full 
discussion to disclose all pertinent facts, misuse of 
criteria, data not hither-to brought out, etc., in order 
to avoid a possibility of unresolved retention of the old 
numbers, the first set of relevance numbers were completely 
discarded by removing them from the blackboard. Each 
member was required to reassess his position and fill out 
a new matrix reflecting his final decision. The second 
ballot was collected for calculation of the expected value 
and variance by the computer. These final data were then 
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inserted at the appropriate cell on the relevance network. 

The discussions o£ the reasoning behind each indi- 
vidual's number assignment and the requirement for the 
committee to reconsider their intitial numbers, has a 
further advantage of drawing the numbers of those with 
limited experience in the subject area into juxtaposition 
with the "expert", thereby weighing his knowledge more 
heavily. However, this will only occur if the "expert" 
can demonstrate his knowledge by his presentation of facts, 
as evaluated in the minds of the individuals. His posi- 
tion and experience will have little impact on the com- 
mittee unless he supports them with up-to-date facts. 

(6) Assignment of Detail Relevance Numbers . Once the criteria 
relevance numbers were decided upon, the decision was made 
about the detail relevance numbers for each of the options. 

A word of caution was given here. There was never to be 
any reference to other criteria at the time of the detail 
number assignment. Each set of data was evaluated within 
the boundaries of the current criterion only, and without 
reference to another criterion and its importance. This 

is very vital to the proper use of PATTERN. 

(7) Group Discussion of the Detail Relevance Numbers . The 
same procedure was repeated with the detail option number 
assignment as was observed with the criteria number assign- 
ment. The policies that were decided upon for this criterion 
and the boundaries of the definition were the limits with- 
in which this discussion was allowed. No reference to any- 
thing bearing on data other than this was allowed by the 

ALJ Associates, Inc. monitor to influence balloting deci- 
sions made at that time. When the monitor felt that there 
was no purpose to be achieved by further discussion, he 
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called a halt, and the group went to the next session. 

(8) Re- Assignment of the Detail Relevance Numbers . As was 

the case in the assigning of the Criteria weights, these 
option relevance numbers were subjected to an intense 
scrutiny by this group of peers from the top level of 
administrative decision-making specialists. It would in- 
deed be unusual if there had not been at least one number 
that was not assigned "in the heat of battle". It is be- 
cause of these circumstances that the complete re-assign- 
ment of numbers was practiced prior to the final acceptance 
of any set. This was usually accomplished by the erasure 
from the blackboard of all of the earlier numbers and the 
reinsertion of everyone's latest evaluation. A short dis- 
cussion was encouraged at this time to assure that the 
same amount of agreement still exists before passing on 
to the other problems at hand. 

It should be observed, that there is never enough 
time to accomplish all of these tasks. There is always 
one more convincing statement that could be made, and one 
more of your peer's opinions that could be "shot down". 

In spite of this, however, one of the double checks that 
was practiced was to select a ballot at random and without 
telling the group about their previously assigned numbers 
to reballot one of the old sections of the data. This 
technique has in the past been found to duplicate the prior 
decisions within an acceptable statistical range as 
determined by no interchange of project order of relevance. 
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d. Relevance Number Calculations 



The fundamental method followed in the assignment of 
numbers is kept consistent throughout the study. The func 
tional matrix used, is represented as follows: 



VARIOUS CRITERIA 
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i th evaluation node option (i * 1, . . . ,n) 
j th criterion under consideration (j = 1, . . . ,m) 
relative value of evaluation option based upon criterion n^ 
importance of criterion nj relative to the other criteria 






VI-34 



. , r(eij) 

1 = 1 



n 



indicates that if all the identified decision 
options are solved, the needs of the deficiency 



1 under consideration are met for the subject 
criteria. 




m 



indicates that all the criteria pertinent to the 
evaluation of the decision options are being 
considered. 



There are various ways that the value of a given decision option 
can be calculated from the data in the matrix, such as expected 
value, most important, expectation- variance and selection of 
aspiration level. Each of these could be investigated on the 
computer to evaluate the sensitivity of the conclusions to various 
calculation procedures. For this study, the continued use of 
the calculation of expected value was considered to be most mean- 
ingful. The expected value for each option was calculated with 
the following equation. 



Since there would be several people inserting data, as will be 
discussed below, a measure of the variance of each option can be 
calculated as well as the uncertainty reflected by each member. 
These variance data could then be subjected to different nonpara- 
metric statistical inference tests that require no knowledge of 
the underlying distribution, to determine the final confidence 
interval associated with the various recommendations. The var- 
iance for each option for each person is. 
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illustrating that his uncertainty with regard to the various 
decision options as well as his assessment of the relative merits 
of the criteria will be included. One way of illustrating how 
the process was followed is to present a hypothetical example. 

EXAMPLE: Assume that the committee has decided that the 

various decision options that are required at 
the project level to meet the stated objectives 
are Planning, Implementation and Analysis. Assume 
the importance of each of these options is to be 
established using the criteria of Organization 
of thoughts and ideas; Completion of each phase 
objective; Collection and development of suffi- 
cient data; and Fulfillment of project expectation. 
Each committee member would be given a matrix 
with the ordinate representing roles and the 
abscissa representing the criteria. Suppose he 
fills it in as follows: 




54 Fulfillment of Project Expectations 
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Note that he determines the relative importance 
of each option, as viewed objectively from this 
relevance node perspective using the Relevance 
Guide Book data previously generated, using one 
criterion at a time, ensuring their total is one. 
Since the only factors he will have to consider are 
the individual criterion at each judgment point, 
he will not have to hazily evaluate the interre- 
lationships of many variables. Note that he will 
not be tempted to use organization or individual 
biases. (This also has a cautionary consideration 
regarding the importance of selecting meaningful 
and comprehensive criteria.) After he has evaluated 
each option using the single successive criteria, 
(order of number assignment is unimportant) , he 
then addresses himself to the relative importance 
(weighing) of the criteria with respect to each 
other, again ensuring their total is one. *The 
exported value and variance of his decisions are 
calculated and entered as shown, but would not be 
done by him in the actual case. 
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A. Introduction 

Tliis section contains the analysis of the relevance numbers 
assigned by the educators and students. The analysis is presented 
according to the various levels of the evaluation network. Tlie 
basic set of computer rankings are also included. 

B. Analysis of Program Level Relevance Numbers 

1. Introduction 

The Program Level of the network is the highest point that 
directly couples the MSL with the PACE program. As was explained 
in Section VI -B, the principle truncation does not require that 
any structuring occur at this level. However, to ensure that 
the MSL was placed in proper perspective, nine other PACE pro- 
grams were selected to comprise with the MSL the Program Level. 

The contribution of these programs to national educational 
objectives was measured in terms of: 

1. technical performance 

2. cost 

3. resource usage 

4. innovation 

The committee of 12 educators assigned weights to the above 
criteria and relevance to the selected programs. 

2. Criteria 

Tlie criteria were selected to reflect the points that a 
decision-maker needs to consider when evaluating programs. They 



are a measure of the degree to which the program contributes to 
accomplishing national educational science objectives. Programs 
were not compared to each other, but rather they were evaluated 
on the degree to which they accomplished the objectives they 
had set for themselves in terms of the criteria. 

To make this evaluation, the educator group was provided 
the "Program Level Relevance Guide Book." The guide book con- 
tained appropriate data on each program. As is noted in Section 
VI. C.4., the Program Level book contained data on only two of 
the four criteria. No data was collected on either cost or 
resource usage. These criteria were defined, however, and were 
assigned weights. The average criteria weights are shown below: 



Criterion 4, innovation, received the highest average rel- 
evance. Most gave it .35 or .40, although one educator assigned 
it a weight of .25. The criterion was interpreted as measuring 
the degree to which the innovative techniques proposed for 
experimentation were in fact developed. The educators felt 
that the principal purpose of PACE funding was to experiment and 
this was their rationale for assigning the weight. 

Criterion 1, technical performance, received the second 
highest weight. All educators assigned it .30 or .35. The 
criterion was interpreted as measuring the degree to which the 
desired behavioral change was exhibited by participants to the 
project . 



1. technical performance 

2. cost 

3. resoruce usage 

4. innovation 



AVERAGE WEIGHT 
.32 

.15 

.18 

.35 



The rationale was that educators are seeking to induce 
behavioral ciiange in students and that programs which induce 
change should be highly rated. There was much discussion about 
rating criterion 1 or 4 highest. Most ranked innovation highest, 
tliree ranked innovation and technical performance equal in im- 
portance and one ranked technical performance highest. The 
group felt that innovation and technical performance should 
count for almost 70^ of the decision in determining program 
value . 

Criterion 2, cost, ranged from .10 to .20 in weight. It was 
interpreted as accurate estimation of costs and economic feasi- 
bility. All educators felt that cost was an important consider- 
ation, but tliat inducing change or attempting innovation was 
much more important. Cost ranked last among the criteria. 

Criterion 5, resources usage, ranged widely from .10 to 
.25. This criterion was interpreted as contribution of community 
resources and staff qualifications to program developemcnt . 

Tliis was considered to be substantially less important than 
innovation . 

Educators felt that experimentating in new techniques was 
most important in a PACE project, followed closely by verification 
of the technique. The educators would credit both highly ex- 
perimental and highly successful programs. 

3. Program Relevance 

At this level, relevance numbers were assigned without 
discussion. This approach was taken because of the nature of 
the level. In making assessments, the group was to compare 
the program only to its own statements as printed in the guide 
book. Allowing no discussion caused higher standard deviations 
than at other levels. 
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The program rankings are shown in Figure. VI I -1 . The 
Mobile Science Laboratory and the Floating Science Laboratory 
ranked one and two. livery educator ranked the MSL best at 
fulfilling its objectives in its own terms. Most educators 
ranked the Floating Science Laboratory second. 

It is clear that the evaluation team found that the 

Mobile Science Laboratory had well fulfilled its objectives. 

•/ 

4. Conclusions 

The educators felt that the most important factors of a 
PACE program are its innovativeness, followed closely by its 
success at demonstrating results. 

The MSL was found to fulfill very well the objectives it 
had established for itself. It was rated particularly high 
on the innovation criterion. 



C. Analysis of Function Level Relevance Numbers 
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1. Introduction 

The basic evaluation performed at the Function Level is an 
assessment of how the MSL enhanced the overall District 241 
science program in the elementary, secondary, teacher and com- 
munity areas. The criteria were selected to measure the value 
of the program use to all participants through augmenting their 
curiosity, knowledge, skills, motivation, attitudes, and general 
familiarity with science, and through increasing the space, 
materials, and equipment available to instruct in laboratory 
science. Evaluation at the Function Level aided in assessing 
the current configuration of the MSL (considering future funding 
would be used for operational costs) and in determing how the 
program might be modified with development funds to meet future 
District 241 educational needs and to provide even greater 
opportunity to use new teaching devices such as the discovery 
method, natural environment and participant self direction. 
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A detailed examination of the factors underlying these numbers 
was conducted. There was no case on any assignment sheet where 
this trend was different nor was there a combination of circum- 
stances that would change this order. On specific criteria, 
as was expected, individuals differed in their order of preference 
but this did not change their final calculation. 

The team found close agreement on the values for the 
elementary and the secondary functional usages, but were somewhat 
less convergent to a unified position for the teacher and com- 
munity usages. The latter relevance numbers are, however, well 
within acceptable tolerances and do not effect the final order 
even when the extreme values are used to calculate relevance. 

2. General Conclusions 

The team concluded that the MSL program of development of 
the laboratory and an appropriate curriculum, followed by 
operational use was a huge success particularly with regard to 
the scientific opportunities offered to the elementary and 
secondary students through direct participation and use of 
facilities. It is also important to continue this basic pro-^ 
gram with more emphasis being placed on the teacher in-service 
training using present facilities. However, the results show 
that the team feels that since the MSL program has achieved a 
considerable degree of maturity and operating efficiency, other 
means exist that should be enthusiastically explored to move 
faster in the direction of further meeting District 241 needs 
and of further providing new opportunities for teaching techniques. 

3. Specific Conclusions 

The evaluation committee felt that the elementary program 
of the MSL has proven to be the most beneficial application (the 
description of the various programs is found in Section V of this 
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report) . Tliere were no complaints on any facets of the operation 
of this part of the program. The elementary principals readily 
scheduled class assignments to place more emphasis on science. 

When the lab is at their school. While elementary teachers made 
little effort to develop specific science units themselves, they 
were able to use with confidence those units developed by a com- 
mittee of teachers directed by two resource teachers associated 
with the MSL. The resource teachers were able to help overcome 
the typical elementary fear about teaching science, since many 
have continued to use MSL study units and materials in their 
science program after the MSL has been moved to a different 
location. These MSL teachers are nearly full-time and are not 
the normal resource support personnel who simply collect science 
materials and books, conduct occasional in-service training or 
act as teacher aids on an infrequent basis. 

It was considered that the majority of elementary teachers 
are rather poorly prepared to teach science. The majority have 
not taken more than the minimal amount of science since most 
decided to specialize in other areas of the wide variety of sub- 
ject matter they are required to teach. The lab would have real 
application to elementary in-service science training. The whole 
educational science outlook of the district could be enhanced as 
far as teacher training is concerned. Little effort has been 
made in this area, however, since the resource teachers have 
spent most of their time either with the students or in preparing 
the units that were being taught. 

There has been a considerable increase in science interest 
among the elementary students. This has been specially noticed 
among girls, possibly due to the women MSL teachers. The students 
demonstrated a much better understanding of science, and the MSL 
has proven to be an important motivative device. The MSL has been 

t 

especially suited to use at the local park where 1,200 students 
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have met with reource people and collected items for study. 

The team felt that the benefit of this activity was in far 
greater proportion to the actual time spent during the trips. 

The children were quite excited about science after each of 
these short visits to the field. Tliere has been a carry-over 
of this curiosity to learning other subjects in the elementary 
classes. The carry over has also been reflected in discussions 
and simple experiments conducted at home as a result of the 
child’s participation in the MSL. 

These same advantages have also been seen at the parochial 
schools which use the MSL on a shared basis. One of the local 
parochial schools will be closing at the end of this school year 
and there is extreme difficulty being created at the other para- 
chial school because of the lack of continuing Federal funds to 
use the MSL to support this school’s program during the next 
year. The program also includes migrant worker’s children when 
they are in the area as well as for special education classes 
in the district. These special classes are for students with 
emotional proble ms or some functional problems such as in 
reading or math. 

One of the basic advantages of the elementary program over 
the secondary program is that the entire student population of 
approximately 4300 students is utilizing the laboratory, while 
in the secondary program approximately 200 students are trained 
each year. It was fully recognized that each elementary student 
got relatively few number of hours of specific individualized 
training, while during the summer program, which lasts roughly 
10 to 12 hours a day during four weeks, each student got much 
more detailed attention during his activities. 

The MSL program supplements the science studies of all 
elementary students and has been in existence for enough time 
for the junior high teachers to see a consistent improvement 
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in the scien . base of the students entering their program. 

This has a direct influence on creating the appropriate atmos^ 
phere for acceptance of the exploratory laboratory-centered 
approach to science teaching using the ESCP and IPS labora- 
tory and investigative curriculum. 

It was strongly felt that in the case of the elementary 
teachers, the lab provided the only support they had in teaching 
their science program. In fact, it was felt that the science 
program in the elementary school would probably be 20 to 50 % of 
the present level without the laboratory. Some of the classes 
would just be offering the book and going through the motions 
during the year. However, with the use of the laboratory, 
teachers are more committed to focus on science. Teachers are 
also given an opportunity to get help from the special teachers 
who have been turned into science coordinators or science con- 
sultants for the district. Now instead of about 30 to 40% of 
the teachers teaching science, the committee felt that about 
80 to 85% of the elementary teachers are involved in the science 
program. One direct reason for this is that the two excellent 
laboratory specialists in the elementary program have become 
science coordinators for the entire District 241, in addition 
to developing in all schools the technological base offered 
by the laboratory. \ 

Vi 

Much work still needs to be done to augment the elementary 
science program. The committee thinks that science cannot be 
taught at the elementary level on a short-term exposure basis, 
such as having a specialist coming in one day a week and holding 
classes. Science should be introduced into the classroom every 
day without waiting for the science lab to come. Also, there 
is considerable expressed interest in using a science program in 
the summer time for the elementary students. 



In the secondary area, the MSL contribution was also 
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dramatic, pointing up the need for the summer program. Tliis 
summer program is an extension of time at a period when science 
teaching can be more effective because of the living environment 
offered by the MSL . The MSL has been demonstrated as a stimu- 
lating purposeful program different from the regular school year. 
It has given the student a block of time in which to pursue a 
particular project without interruption of bells and jumping from 
one subject to another- -an opportunity to follow an interest of 
his own choice with the help of special teachers, equipment, and 
scientific tools and techniques. There is a need to enhance the 
understanding of the interrelationships that occur in field 
situations. (The specific benefits of this program in terms of 
the growth of the student capability through participating in 
successive years v>rill be discussed in the following section.) 

This payoff tends to come in the change of attitude that is 
occurring for various people in terms of better relationships 
with their children, new programs in the district, and better 
grades for their class. It was felt that this district is 
basically a conservative one, yet it is not doing conservative 
things. The only real variable in the science arena has been 
the MSL. Therefore, it appears that it would have to be con- 
sidered very strongly as the casual thing that is changing some 
of the attitudes and ideas. 

In the secondary area, teacher training has been enhanced by 
involving the people in a formal one-week job course. However, 
those that participate during the entire summer are receiving 
significant in-service training which is having quite an impact 
on their program in the school. The living philosophy that the 
program gives them reinforces the attitude change started in 
the week-long course, but the actual attitudes of the teachers 
shift radically through living with the students in the field 
on the program. 
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It was felt that the typical secondary teacher fallaciously 
believes he does not need in-service training while the elementary 
teachers indicate a need even in fields of their specialties. 
Efforts have been made in some of the local teacher colleges to 
have their students practice teach in the lab. These overtures 
have to date been rejected for a variety of reasons including 
tight scheduling. The kind of in-service training that the MSL 
could offer would be techniques of working with children and with 
other people. This is something every teacher should want to 
know more about. 

The team feels that considerably more emphasis should be 
placed in the area of teacher training, especially in support of 
the elementary program. They felt quite strongly in their 
relevance assignment that the introduction of the laboratory 
activities in the elementary level was building a long-term 
base for enhancing the total scientific capability that would 
permeate through the secondary schools to the community. The 
formal in-service training for the teachers in the MSL program 
has been minimal. However, by the participation of the teacher 
during the school period, when their students were part of the 
laboratory, greatly augmented their knowledge of science. The 
committee feels that considerably more effort should be spent 
in attempting to utilize the laboratory for development of new 
techniques, while continuing to work on the same basic techniques 
with the current program. Considerable emphasis should be placed 
on developing the teacher relationship with children rather than 
training in very specific subject matter in a traditionally 
oriented way, such as learning more subject matter or getting 
more ideas to do a more effective job in the classroom. In the 
secondary area, where the teachers have the skills and the 
subject matter knowledge, they tend to focus on environmental 
relationships with students. The laboratory at the secondary 
level gives a teacher the opportunity to live with the student 
for three weeks in the field. The teachers attitude toward the 
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student is completely different than it was before. The student 
is no longer a thing, he is a person. This is indeed a unique 
type of in-service training that is being offered. 

The group felt that considerable work should be done in the 
area of community activities to increase local awareness of the 
innovative capabilities that exist and of the public image that 
the Albert Lea district has achieved at the national level. This 
is evident both by the many organizations wanting to borrow the 
laboratory and by those who visited it wanting to emulate its 
operations and to identify the problems associated with it. 
Considerably more work should be done toward forming a strong 
core of directors, leading citizens in the community that would 
take an interest in the over-all scientific program and aid the 
school program in continuing the laboratory and expanding its 
innovative characteristics. This public relations spirit could 
be culminated in a series of special adult education classes, 
that if characteristic of other school districts, would have a 
tendency to enlarge in their applicability. 

The Albert Lea district does have several opportunities for 
formal adult education classes and the group feels that there 
will continue to be a great drift toward the vocational area. 
There is a state vocational school being planned as well as 
an existing adult school that deals with hobbies and vocational 
training. However, there has been no effort in the pure science 
area. The group was unable to be as specific in their consid- 
eration in this area because of the lack of experience and this 
is reflected in the higher per cent relevance deviation, in: 
terms of their agreement with respect to the type of new 
investigations that should be conducted for the community 
level. However, the strong feeling was that the program is now 
developed to a point where considerable time could be devoted to 
this area. No one wished to sacrifice the additional work that 
should be done in the basic program especially in the elementary 
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study activities for the community activities. 

The group felt very strongly that considerable effort 
should be placed in developing a new innovative program use 
for this laboratory. Some of the opportunities discussed in 
detail were such programs as equipping the laboratory as a 
math -computer facility for software training, equipping the 
lab in the winter for a special kind of program that couldn't 
be put in the school- -such as a cold weather ecology study to 
be used for teacher training aid as well as for students, 
creating teaclier interest and student interest in new scien- 
tific capabilities using techniques in the field of advanced 
astronomy, television-communication electronics, study of the lakes 
and agricultural investigations. For example during the winter in 
addition to having the laboratory placed beside an elementary 
school, it could be equipped with a special type of instrumentation 
and pulled next to a high school where the gifted students and 
highly motivated teachers could come together and do advanced 
studies in science, covering fields that they would mutually 
agree upon, and augment the various vocational clubs that are 
already present in the district. 

Much more work can be done in the integration of the laboratory 
with the recently founded college by calling upon the teachers and 
student trainees of that school to assist in the program. To 
achieve these goals we strongly recommend that the laboratory 
program director be relieved of some of his classroom duties so 
that he can use the same creative talent and experience demonstrated 
throughout this excellent program to build into it even greater 
capabilities for the science program of the district. 
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D. Analysis of Project Level Relevance Number 

1. Introduction 

The Project Level of the network consisted of a set of 99 
student projects conducted under the Secondary Program of the 
MSL. The education? 1 value accrued to the students by con- 
ducting these projects in the environment offered by the MSL 
was assessed in terms of their performance based on: 

1. The best use of time. 

2. Flexibility in coping with situations beyond the students 
control . 

3. Organization of the study approach. 

4. Production of meaningful results and conclusions. 

5. Use of supplemental resources (students only). 

6. Educational value. 

A committee of 12 educators and 12 groups of four students 
each independently assigned weights to the above criteria and 
relevance numbers to the student projects. 

2. Criteria 

The criteria were selected to enable comparison of projects 
to each other. They are a measure of enhancement of student 
capability from participation in the MSL. By comparing projects 
to each other rather than an absolute scale, evaluation of the 
relative impact of the MSL can be made. Following the standard 
procedure of comparing students who use the MSL to a control 
group who had not was not possible. No control group was 
established during the program since this would have been of 
little value as the MSL group was exposed to many more factors 
than could be used for the control group. Measuring progress 
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from one student group to another, both of which had been 
through the MSL developed as the logical solution. 

The weights assess the importance of the criteria in evalu- 
ating the projects. Criteria may be added or deleted at any time 
during the assignment session. In this fashion a complete relevant 
set of Judgment factors is created. 

The average criteria weight for both educators and students 
are shown below. 



EDUCATOR STUDENT 

AVERAGE WEIGHTS AVERAGE WEIGHTS 







Original 


Normal i zed 


1. Best Use of Time. 


.171 


.154 


.171 


2. Flexibility in coping the 


.130 


.118 


.131 


students control 








3. Organization of the Study 


.282 


.214 


.237 


Approach 

4. Producing meaningful Results 


.265 


.218 


.242 


and Conclusions 








5. Effective Use of Supplemental 




.109 




Resources 








6. Educational Value 


.152 


. 198 


.219 



Criterion 5, supplemental resources, was not used by the 
educators in assigning weights. The group felt that the criterion 
was not important to their assessment. The student relevance re- 
flects the redistribution of their assessment of Criteria 5 to 
the other criteria for the purposes of making a comparison of 
weights. Student relevance rankings, of course, include the 
criterion . 

The above results show that on the average students and 
educators agreed on the weight of criterion 1 and 2. They dis- 
agreed widely on criteria 6 and 3, and were fairly close on 4. 
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However, the data that lies behind the average figures shows 
clearly the meaning of the criteria. 

For the educators, it is notable that there was the least 
disagreement in the values for criterion 3, the most important, 
and criterion 2, the least important to the decision. Criterion 
3, organization of the study approach was judged to be the most 
important because it measured the whole of the project. The 
study approach entailed planning, implementation and analyzing. 
This criterion compared the projects on the degree to which 
these three parts of the project were completed. From this, one 
may infer that to educators, the most important feature of the 
secondary MSL program is the experience gained in working out an 
orderly approach to a problem. Every educator on the committee 
gave this criterion a weight of .25 or .30 with the average being 
. 282. 



In contrast, every educator gave criterion 2, flexibility in 
coping with situations beyond the students control, either .10 or 
.15 with the average being .130. This criterion measured the 
adaptability of the student in readjusting his plans and his 
project to the unexpected. Adaptability, one may infer, is the 
least important feature to be gained from the MSL experience. 

On criterion 4, production of meaningful results and con- 
clusions, the educators were in fair agreement. Most gave it 
.25 or .30, but two felt that it should be .20 and one, .35. 

These two extreme positions represented disagreement about the 
objectives which one pursues in educating. One would stress the 
methods and ways of attacking a problem and the other would 
stress reaching conclusions about the problem. The average was 
.265. 



On criterion 1, best use of time, the educators were again 
in relative agreement. Most gave the criterion .10, .15, or .20 



but one felt the value should be .25 and one, .30. This cri- 
terion judged what might have been done on this project against 
what was actually done. The determination of what might have 
been done was made in the student's terms. The extreme position 
on this criterion felt that the development of the best possible 
project was the overriding factor in evaluation projects. 

The educators disagreed most in criterion 6, educational 
value. The weight varied from .00 to .25 with the average being 
.152. The extreme positions represented disagreement about 
assessing the applicability of measuring future benefits to the 
student from his MSL experience. One position believed that 
future benefits were irrelevant to evaluating student projects. 
The other felt that some projects would have more future benefits 
than others and that this fact was highly relevant to evaluatii^ 
the MSL experience. 

The educators assigned criterion weights in one group of 12. 
Students assigned weights separately in 12 groups. This means 
that one would expect more disagreement in the student results. 
This in fact is the case. 

The students' results, however, paralleled to a degree the 
educators' results. On all criteria except 6, the students were 
in fair agreement, that is, the ranges were within statistically 
acceptable bounds. In those five criteria, there were in general 
a few at the very high end of the scale and a few at the very low 
end with the majority of assignments concentrated near the 
averages. The students agreed with the educators that adapt- 
ability was the least important benefit to be gained from the 
MSL program. It is interesting to note that the students felt 
that reaching meaningful conclusions was more important, although 
only slightly, than experience in approaching a project. This, 
one may infer, indicates that educators and students are not in 
complete concurrence on what educational objectives should be. 
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Devclopiiu; tlic best possible project was the same weight by 
both educators and students. 

This leads us to criterion 6, educational value. The 
student distribution of weights clearly separated into two parts. 
One set of 12 students gave the criterion the very high average 
weight of .37. The other set of 35 students gave it a low 
average weight of .13. One-fourth of the students felt that the 
future benefits gained from doing the projects were the most 
important consideration . It appears that students and educators 
alike had difficulty in reaching an agreed position on this 
criterion. 

Distributions of Teacher and Student Criterion 6 Weight 
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Nine of the educators on the evaluation committee conducted 
the student assignment sessions. It is interesting to note that, 
while some of the educator criteria rankings had very high cor- 
relation with that of the student group, they concluded the aver- 
age correlation was .69 which is below the threshold of signifi- 
cance of 5^. The correlations ranged from .15 to .975 (1.00 would 
mean perfect match) . This demonstrates that the educators did 
not bias student criteria weight assignment. 

For more details on the criteria see section VI. C.3. 

3. Project Relevance 

The relevance number is a measure of the educational content 
of one project with respect to the others. Both educators and 
students ranked the 99 projects. There was a rank correlation 
of .76 which is significant at the 1 % level. A partial correla- 
tion was calculated for the first, middle and last thirds of the 
list. This yielded correlations of .83, .71 and .73 respectively 
which are also significant at the 1 % level. This demonstrates that 
students and teachers had the same basic understanding of the cri- 
teria by which projects were measured. One may infer also that it 
was easier to rank the better projects and that the poorer projects 
were slightly easier to rank than those in the middle of the sample. 
The relevance rankings were from highest relevance to lowest. 

Analysis of the relevance at the Project Level involved 
determination of the relationships of project relevance on two 
parameters: student grade level and student project phase. 

The relevance by grade level yielded the average relevance 
per project shown below where a distinct trend of increased 
relevance exists toward the higher grade. 



GRADE 



AVERAGE RELEVANCE 



7 

8 
9 

10 

11 



.0059 

.0088 

.0109 

.0197 

.0255 
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The greatest increase in average relevance may be seen be- 
tween the ninth and tenth grades. This is undoubtedly due to 
the maturing of the student in his awareness of science and to 
his increased preparation in science. The results below show 
an equally meaningful interval between the 7th and 8th grade. 



The mean decile is the average group of ten into which 
the weighted average of the project ranks falls, e.g., 1 means 
in the first ten, 2 in the second ten, etc. 

Student grade level had an impact on the relevance his 
project received. However, it should be noted that the highest 
ranked project was a group project done by two 10th and 11th 
graders. The highest individual project was done by a 10th 
grader. The highest and lowest ranked projects by grade are 
shown below: 



GRADE 



MEAN DECILE 



7 

8 
9 

10 

11 



7.5 

5.5 
4. 
2 . 
1 . 



HIGHEST FOR GRADE 



RANK (99 Lowest) 



7 

8 
9 

10 

11 



31 

15 

8 

2 

5 



LOWEST FOR GRADE 



RANK (99 Lowest) 



7 

8 
9 

10 

11 



99 

91 

88 

59 

5 



n 
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The highest seventh grade project was higher than about 
one-half tlic ninth grade projects, and it was higher than 
one 10th grade project. 

r 1 

I i 

I 

Project relevance by student phase (Basic, Phase I, II, and 
III) yielded the average relevance per project shown below. 



GROUP 


AVERAGE RELEVANCE 


Basic 


.0063 


I 


.0075 


II 


.0094 


III 


.0170 



U 



i j 

Ll 

u 

u 

u 



Again the results show that student ability to plan, to 
collect samples and to analyze data improves with increase 
student participation in the MSL program. It is significant to 
educators working for better methods of implementing the learning 
process that immediate and continued improvement occurs for 
students using the MSL. The highest seven projects were all 
done by Phase III students and the lowest seven were all by 
Basic Phase students. 



The highest and lowest ranked project from each phase are 
shown below: 



HIGHEST FOR GROUP 
Bas ic 
I 
II 
III 

LOWEST FOR GROUP 
Basic 
I 
II 
III 



RANK (99 Lowest) 
13 
21 
8 
1 

RANK (99 Lowest) 
99 
91 
88 
59 



u 

y 
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The highest project from the Basic Phase was done by a 
ninth grader, ranking thirteenth in the list. This shows MSL 
program potential for early achievement as well as for 
progressive improvement. 

Combining and comparing the data for grade and for phase 
yielded the results below: 

GRADE PHASE AVERAGE RELEVANCE 



8 


I 


.0075 




II 


.0129 




III 


.0099 


9 


I 


.0080 




II 


.0116 




III 


.0128 



The only available data occurred for eighth and ninth 
graders. Seventh graders did only Basic Phase project and 
tenth graders did only Phase III projects. 

The trend in the table is that of progressive improvement 
toward the upper phases. The one exception consisted of the 
one Phase III eighth grade project included for completeness. 

The results in this table certainly confirm the benefit to the 
students of participation in the MSL program. 

4. Conclusions 

The evaluation found that the most important thing to be 
gained from the MSL is exposure to ways of attacking and 
solving problems. Production of meaningful results and conclu- 
sions was also found to be important. On this basis it is clear 
from the relevance analysis that the MSL secondary program had a 
significiant learning impact on the students who participated in 
it. 



All tlic data showed that students who had progressed through 
the four pliases of the program were measurably effected by their 
experience. Although grade level correlated strongly with ben - 
fits, the comparison of eighth and ninth graders showed that bene- 
fits from participation in the MSL program increased for every 
year the student was in the program. 

The value of the MSL is confirmed by these results. Com- 
parison of similar groups with similar backgrounds differentiated 
principally by more exposure to the MSL presents conclusive 
evidence of the MSL impact on increasing educational benefits 
to the students through working on projects. 
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E. Analysis of Means Level Relevance Numbers 

1. Introduction 

The Means Level of the network consisted of the three 
phases of each student project. The educational value to the 
students of planning, implementing and analyzing in their 
projects through use of the MSL program was assessed in terms 
of : 

1. Organization of thoughts and ideas. 

2. Collection and development of sufficient data. 

3. Completion of the three phases of the problem, as 
defined by the students. 

4. Fulfillment of phase expectations. 

5. Effective use of the MSL program (teachers only). 

A committee of 12 educators and 12 groups of four students 
each independently assigned weights to the above criteria and 
relevance numbers to the student projects. 

2. Criteria 



The criteria were selected to enable comparison of the 
project phases to each other. They are a measure of the en- 
hancement of student skills in planning a project, collecting 
data, and reaching a conclusion. Comparing the phases to each 
other will apprise the student of his relative strengths and 
weaknesses in conducting a science project. The weights assess 
the importance of the criteria. The criteria represent the 
complete set of factors on which to measure the three parts 
of each project. 



The average criteria weights for both educators and students 
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are shown below: 

EDUCATOR STUDENT 





AVERAGE 


WEIGHTS 


AVERAGE WEIGHTS 


1. Organization of Thoughts 


Original 

.233 


Normalized 
. 280 


.330 


and Ideas 


2. Collection and Develop- 


.156 


.308 


.307 


ment of Sufficient Data 


3. Completion of Each 


.182 


. 219 


. 179 


Phase Objective 


4. Fulfillment of Project 


.160 


.193 


.184 


5. Use of MSL Resources 


.169 


* 


#c 



*Criterion 5, use of MSL resources, was not used by students 
in assigning criteria weights. This criterion was added by the 
educators who wanted to relate the applicability of doing project 
sections in the MSL. To allow comparison, the weight of criterion 
5 was distributed to the other criteria. Educator relevance 
rankings include the criterion. 

The above results show that on the average educators and 
students agreed on the weight for criterion 2. They were in 
wide disagreement on criteria 1 and 3, and were fairly close 
in 4. The data behind these average values clearly shows the 
meaning the criteria convey. 

Educators judged criteria 1 and 2 most important in mea- 
suring student achievement. Criterion 2, collection of suffi- 
cient data, was given .256, the highest average weight. 

Educators did not reach close agreement on this criterion. Most 
assigned a weight from .20 to .30, but two gave it .35 and one 
.15. These extremes represented differences in emphasis in 
collection. The one position stresses learning ways of col- 
lecting data, planning the collection and implementing it to a 
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sufficient degree to make an analysis. The other would stress 
methodology, irrespective of data. 

Criterion 1, organizing thoughts and ideas, was ranked 
second to criterion 2 by most educators. It was assigned an 
average weight of .233. This criterion assessed the degree to 
which the student developed a plan of action and stated an 
analytic methodology. Four educators ranked this equal to 
criterion 2 . Two ranked it first, while three ranked criterion 
2 first. There was again a wide range on weights, .15 to .30 
with most educators assigning .20 or .25. The extremes again 
reflect the difference between stating a methodology and col- 
lecting data. 

Criterion 3, completion of each phase objective, evoked 
substantial disagreement. Two educators weighed this .30, 
while the rest gave it .10 to .20. The extreme here reflected 
the belief that finishing something deserves significant con- 
sideration in determining program benefits to students. The 
average weight was .182. 

Criterion 5, use of the MSL, split the group in two parts. 
One felt that .10 to .15 was the appropriate weight, while the 
other felt that .20 to .25 was most relevant. This criterion 
measured student necessity to use MSL for his project. It 
measured the degree of creativity for the student in conceiving 
his project as something that could not be done at home. 

The students were in closest agreement with each other on 
criterion 2, collection of data, which they ranked second with 
an average weight of .307. Student weightings ranged from .10 
to .60 with slightly less than half the students giving the 
criterion a weight greater than .35. This compares with 
criterion 1, organization of thoughts, which was ranked first 



o 



with an average weight of .330. Here again slightly less than 
half the students gave the criterion a weight greater than .35, 
while the range was from .10 to .15. These figures reflect the 
division among students, similar to that among educators, as to 
the necessity for creating a methodology or for collecting data 
on which to operate. In any case, both students and educators 
believe that developing a methodology and collecting data are 
by far the most important items to consider in designing 
and implementing a project. 

The students found fair agreement in the weight of cri- 
terion 3, completion of phase objectives, assigning an average 
value of .179, the lowest of the criterion weights for students. 
They felt that finishing the phase was not really as important 
as its organization. 

Criterion 4, fulfillment of project expectations, was 
assigned average weight of .184. Some students weighed this as 
high as .40, but more than three quarters weighed it below .20. 
Students felt that fulfillment of the goals they had set for 
themselves was slightly more important than those set by the 
project. Educators felt the opposite, ranking self-set goals 
substantially lower than project-set goals. 

The educator and student criteria weight average rank 
correlation was .52, well below the threshold of significance 
of 5%. This again shows that educators had no influence on 
the assignment of criterion weight by students in the group 
the educator monitored. 



For more details on the criteria see section VI. C.3. 
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3. PI A Relevance 

The relevance number is a measure of the accomplishment in 
one phase of the student project against the others. Both 
educators and student assigned relevance at the PIA level. 

The ranked listings were from highest to lowest relevance. 

Analysis of the relevance numbers at the Means Level in- 
volved determination of the relationship of planning, implemen- 
tation and analysis relevance on two parameters: student grade 
level and student project phase. A comparison of project 
phase relation to grade level was also made. 

The average relevance by grade level yielded the average 
relevance per PIA means per project as shown below: 



These results indicate that higher grade levels on the 
average recieve higher relevances. The eighth and ninth grades 
received slightly higher scores than did the group projects 
(which combined students from several grades) . The seventh 
grade was lowest in the planning and the analysis sections, and 
was about the same as the group projects in the implementation 
phase. The tenth and eleventh grades received the most rel- 
evance in all three of the project sections. 



GRADE 



AVERAGE RELEVANCE 



Groups 



7 

8 
9 

10 

11 



P I A 

0026 .0021 .0011 
0044 .0025 .0018 

0049 .0038 .0025 

0072 .0067 .0054 

0082 .0078 .0083 

0041 .0020 .0021 
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The results also indicate that, in general, planning received 
higher relevance followed by implementation and finally analysis. 



This indicates that on the average students are planning their 
projects better than they are collecting or analyzing data. 

The following shows the highest ranked and lowest ranked project 
section by grade level. 

HIGHEST FOR GRADE RANK (298 Lowest) 




P 


I 


A 


7 


48 


80 


159 


8 


41 


45 


88 


9 


12 


23 


33 


10 


5 


3 


8 


LOWEST FOR GRADE 


RANK 


(298 


Lowest) 




P 


I 


A 


7 


298 


289 


297 


8 


212 


268 


286 


9 


168 


290 


292 


10 


119 


173 


188 



The same sequence of planning followed by implementation 
and analysis is evident. The grade level of the student had 
an impact on the relevance in each category. 

The average relevance for planning, implementation and 
analysis by student phase is shown below. 



PHASE AVERAGE RELEVANCE 





P 


I 


A 


Bas ic 


.0027 


.0022 


.0012 


I 


.0039 


.0020 


.0014 


El 


.0045 


.0030 


.0019 


III 


.0061 


.0058 


.0047 
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The more advanced students in the MSL program receive the 
higher relevances. The differential is not as great as for 
grade level, but the evidence is present in every category. It 
is interesting to note that in implementation and analysis, only 
Phase III shows a substantial difference from the other student 
phases. Planning shows a much more differentiated range. Again, 
the categories are ordered from planning to implementation to 
analysis . 

Extracting the highest and lowest rank in each student 
phase for planning, implementation and analysis yields. 



HIGHEST IN GROUP 


RANK 


(298 : 


Lowest) 




P 


I 


A 


Bas ic 


40 


31 


159 


I 


32 


64 


107 


II 


10 


23 


33 


III 


1 


2 


4 


LOWEST IN GROUP 


RANK 


(298 


Lowest) 




P 


I 


A 


Bas ic 


298 


289 


297 


I 


212 


268 


295 


II 


182 


290 


296 


III 


119 


250 


210 



These results verify the trend toward higher relevance 
from Basic to Phase III in relevance rank. The most striking 
thing that is to be found in the "Highest in Group" section is 
the fact that the highest Basic Phase project was ranked higher 
in the implementation section than was that from Phase I. This 
seeming discrepancy is explained by the fact that this project 
was done by a tenth grader. 
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Cross -tabulating grade level and project phase yields the 
results shown below. Again the seventh and tenth grades have 
not been included because of their concentration in one phase. 



GRADE 


PHASE 


AVERAGE RELEVANCE 






P 


I 


A 


8 


I 


.0039 


.0019 


.0014 




II 


.0049 


.0036 


.0025 




III 


.0045 


.0026 


.0026 


9 


I 


.0031 


.0029 


.0018 




II 


.0048 


.0033 


.0021 




III 


.0051 


.0044 


.0031 



The results in this table show that the MSL program had a 
significant effect in each category of the student project. 

The relevance in every case strongly emphasizes the fact that 
the MSL enhanced student capabilities between Phase I and Phase 
III. The differences between Phase II and III in the eight 
grade is likely due in fact to the small sample size as was 
mentioned in the project analysis. Also the ranking of plan- 
ning followed by implementation and anlaysis holds for this 
cross -tabulation. 

The MSL had significant impact on the secondary student in 
enabling him to better prepare a project. The planning section 
received higher relevance than the implementation or the an- 
alysis section for almost every student. Cross - tabulation 
between grade level and project phase showed that the projects 
in Phase III received more relevance than those in Phase I. 

The data shows that the analysis and the implementation 
sections improved in relation to the planning section. Students 
started in the program developing the planning phase by far the 
best. After four years in the MSL program, understanding of the 
analytic and collection methods increased making the overall 
project much better balanced in Phase III. 
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4. Conclusions 

The evaluation found that the most important factors in 
determining student strengths and weaknesses in conducting a 
science project are the creation of a methodology and a plan of 
action and collection of sufficient data with which to work. 
Students and educators both agreed that these were the key 
factors . 

It is clear from the data that longer exposure to the MSL 
yields increased benefits to students. V/hile the data showed 
that grade level correlated strongly with relevance, the 
cross “ tabulation of grade level and project phase strongly 
demonstrated that benefits from the MSL program increased 
every year. These results are strengthened by the fact that 
in each project section relevance increased from the Basic 
Phase to Phase III, with grade level held constant. 

The data also indicates that although students were better 
able to plan than to implement and to analyze, those students 
who had progressed through all four phases of the MSL were 
able to develop a more balanced science project in terms of 
accomplishment in each of the project sections. The students 
were better able to develop a methodology for each project 
section and were better able to collect the required data. 

The value of the Mobile Science Laboratory for science 
education is confirmed by these results. Comparison of some 
grade level students with similar backgrounds differentiated 
principally by more exposure to the MSL presents conclusive 
evidence of MSL application in providing expanded science 
education benefits. 
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F. Sample of Computer Listings 

This section includes a basic set of computer runs used 
in the analysis. Due to the extensive volume of computer output 
listings, the entire array of runs cannot be included in this 
report. However, a complete copy of computer listings was 
provided tp the Project Director. 

Figure VII -1 is the educator ranked relevance number 
listing of the program level. The left-most column is the rank 
numbers (0001-0010) followed by the option IDs, the option titles 
and the branch relevances. 

Figure VI I -2 is the educator ranked relevance number 
listing of the function level. The left-most column is the rank 
numbers (0001-0004) followed by the option IDs, the option titles 
and branch relevances. 

Figure VI I -3 is the educator ranked relevance number 
listing of the project level. The listing is composed of the 
ranking (0001-0099) in the left-most column followed by the 
option IDs, titles and branch relevance respectively. 

Figure VI I -4 is the student ranked relevance number 
listing of the project level. The listing is composed of the 
rankings (0001-0099) in the left-most column followed by the 
option IDs, titles and branch relevance respectively. 

Figure VII -5 is the educator ranked relevance listing (first 
25 and last 25) of the planning, implementation and analysis 
level. The listing is composed of the rankings (0001-00025... 
0270-0294) in the left-most column followed by the option IDs, 
titles and branch relevance respectively. 
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Figure VII -6 is the student ranked relevance listing (first 
25 and last 25) of the planning, implementation and analysis 
level. The listing is composed of the rankings (0001-0025... 
0270-0294) in the left-most column followed by the option 
IDs, titles and branch relevance respectively. 
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Figuic VII • 6 Student Branch Relevance Listing of the PIA Level (2 of 2) 



VIII. APPENDICES 

A. Study Personnel and Their Comments 

l. Introduction 

The evaluation team consisted of a balanced interdiscipli- 
nary group of students, educators, administrators and manage- 
ment personnel. This section includes the resumes of all 
adult participants and a listing of students by name, grade and 
school. The educators and students were asked to submit their 
comments and criticism on any facet of the Mobile Science Lab 
program and the Evaluation Phase. All responses are included 
verbatum (with only spelling errors corrected.) 

All educator participants in the balloting sessions were 
an integral part of the entire evaluation from inception to 
completion. The student comments are, of course, from a widi 
cross-section of program participation in terms of years in 
the program and group level of experience. 

The evaluation contained a high degree of emphasis on 
measuring the scientific value of the Mobile Science Laboratory 
program, but it was well recognized that there are many other 
facets that reflect on the efficacy of an objective analysis. 
Examination of the array of talent represented by the evaluation 
team will illustrate that it was well rounded and qualified to 
make assessments in all phases of the evaluation. Tlieir interest 
and dedication to this excellent program is clearly evident in 
the sincere comments of both educators and students. 
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2. ALJ Associates, Inc. Resumes 

M/i. A. L. Jd^tlcQ. 

Mr. Jestice studied at Denver University, University of 
Minnesota, and received his degree in engineering/operations 
research from George V/ashington University. He has studied 
extensively in the fields of decision theory, economics, 
psychology, and international affairs. He has been deeply in- 
volved in education needs while serving in various roles in the 
PTA and in teaching. He has been called on as an operations 
research specialist to give courses, guest lectures, etc. for 
the University of Minnesota, UCLA, Federal Management Confer- 
ence, Washington Operations Research Council, George Washington 
University, Brookings Institute, DDR§E, the Institute of Manage- 
ment Sciences, Operations Research Society of America, Army 
Advanced Management Training Schools et al. Mr. Jestice is 
founder and President of ALJ Associates, Inc. 

M/i. J. L. Klfik 

Mr. Kirk received his B.S. in politics, economics, and 
engineering from the Massachusetts Institute of Technology in 
1964. He is in the process of writing a thesis to complete the 
requirement for M.S. in operations research at George Washington 
University. Mr. Kirk has worked in systems analysis for five 
years with Honeywell, Inc., the U.S. Government and ALJ Asso- 
ciates, Inc. He is presently an Associate with ALJ Associates, 
Inc. 

M/i. C. A. TayZoK 

Mr. Taylor received his B.A. in 1945 and M.A. in 1955, both 
in sociology and statistics from the University of Florida. He 
is studying for his Ph.D. in operations research at American 
University. Mr. Taylor has 25 years experience in systems an- 
alysis covering a broad spectrum of education activities in- 
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eluding teaching at the high school and college level. Me has 
done development work in mathematical modeling and simulation 
for Naval Aviator Training, Bio-Medical Statistical applications 
and in criminology studies. He is presently an Associate with 
ALJ Associates, Inc. 

Mr. Ostrich received his B.A. in 1950 and M.A. in 1953 both 
in psychology from George Washington University. He is now com- 
pleting his doctoral dissertation. He has received his certif- 
icate of doctoral candidacy for his Ph.D. in history from the 
University of Pittsburgh. During his 20 years of research ac- 
tivities in education and medical arenas Mr. Ostrich has per- 
formed extensive studies in improvement of perception capabil- 
ities, common skills and knowledge of individuals. He is 
presently an Associate with ALJ Associates, Inc. 

Mh., a. a. Hunt 

Mr. Hunt attended Trenton State College, Northern Virginia 
University and Electronic Computer Programming Institute of Vir- 
ginia where he graduated summa cum laude as a programmer, and 
is now enrolled in George Washington University. Mr. Hunt has 
considerable experience with the application of the PATTERN 
methodology on various computers. He is presently an Associate 
with ALJ Associates, Inc. 

M/l-6 . £. W. KtA.k 

Mrs. Kirk received her B.S. in French from the University of 
Georgia in 1965. She has also pursued course work towards an 
M.A. in education at the University of Maryland. Mrs. Kirk has 
taught secondary school for four years, presently French at 
Randolph Junior High School in Rockville, Maryland. 
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3. Educators 

a. Resumes 

S. R. B^eckmA 

Mr. Breckner recieved his B.S. in education from Winona 
State College in 1946. He has studied further at the graduate 
level in mathematics and science, his most recent experience 
being at the Introductory Physical Science Institute, Nebraska 
Wesleyan University. He has taught in secondary schools for 
27 years, presently teaching introductory physical science at 
the eighth grade level in Southwest Junior High School in 
Albert Lea. 

R. C. CtoLfik 

Mr. Clark has done considerable work toward a doctorate 
degree with credits from Stanford, University at Redlands, 
and San Francisco State college. He is the new State Science 
Consultant for Minnesota. Mr. Clark came to Minnesota from 
Lompac , California where he was science supervisor for the 
elementary schools. He has taught biology in high school 
during which time he developed and taught a course in marine 
biology. 

Ma, Gzofigz Fenze/te 

Mr. Denzene recieved his B.S. in Social Studies from Uni- 
versity of Minnesota in 1942. He recieved his M.A. in history 
from University of Minnesota in 1947. Mr Denzene has also done 
other graduate level study including summer sessions at North- 
western University and Ohio State University. He has taught 
in secondary schools for 20 years and is now teaching U.S. and 
world history at the tenth and eleventh grade level in Albert 
Lea . 
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M/l. I/. L. Vo6 6 

Mr. Doss recieved his B.S. in Social Studies from Mankato 
State College in 1953. He recieved his M. Ed. in English from 
Mankato State College in 1961. Mr Doss has taught secondary 
school for 13 years, his present assignment being English and 
social studies at the seventh grade level at Southwest Junior 
High School in Albert Lea. 

, Robe.A.t EntoAi 

Mrs. Entorf received her B.S. in Education from Iowa State 
University in 1946. She has also pursued various graduate level 
studies, her most recent experience being elementary science 
methods at Mankato State University. Mrs. Entorf has taught sec- 
ondary school for 6 years and is presently one of two Elementary 
Science Resource Teachers assigned to the Mobile Science Labora- 
tory . 

Ma, E. EAlck^on 

Mr. Erickson recieved his B.S. in Business Education from 
Mankato State College in 1951. He recieved his M.S. in Business 
education in 1960 from Mankato State College. Mr. Erickson has 
also pursued various graduate level studies. He has taught sec- 
ondary school for 18 years, his present assignment being business 
education department charirnan at senior high school in Albert Lea. 

Ma. R. HaAdXng 

Mr. Harding recieved his B.E. in education from Bemidji 
State College in 1939. He recieved his M.A. from the University 
of Minnesota in 1956. He has also pursued various graduate 
studies. Mr. Harding has 28 years experience teaching at the 
elementary and secondary level. Me is presently teaching earth 
science at the ninth grade level in Albert Lea. 
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M. KyZZo 

Mr. Kyllo received his B.S. in Agriculture from University 
of Minnesota in 1951. He has also pursued graduate studies in 
conservation and biology. He has taught 17 years in adult and 
secondary education. Mr. Kyllo is presently teaching biology 
at the seventh grade level in Albert Lea. 

M/l. M. 

Mr. Lawrence received his B.S. in education from Mankato 
State College in 1961. He received his M.A. in elementary ad- 
ministration from Mankato State College in 1964. Mr. Lawrence 
has taught 8 years at elementary level. He is presently the 
principal at Lakeview Elementary School. 

MonZqud 

Sister Monique recieved her B.S. in education from College 
of St. Teresa in 1960. She has also pursued graduate studies 
most recently at Bradley University in science. Sister Monique 
has taught 7 years at the elementary and secondary level and is 
presently principal at St. Theodore Grade School in Albert Lea. 

M/l. K, V , ?zdzxi>on 

Mr. Pederson recieved his B.S. in social studies from Winona 
State College in 1957. He received his M.A. in education adminis- 
tration from the University of Minnesota in 1961. Mr. Pederson 
has taught secondary school for 5 years and has been an adminis- 
trator at secondary level for 6 years. He is presently senior 
high school principal in Albert Lea. 

R. SakmZdt 

Mr. Schmidt received his B.A. in science from Augustana 
College in 1949. He has also pursued graduate studies, most 
recently mathematics at Kansas State University. Mr. Schmidt 
has taught 16 years at the secondary level and is presently 
teaching mathematics in Albert Lea. 
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b. Educator Evaluation Comments 

Teacher No, 1 

"1. Criticism--Pressure of time (more time needed and drawn 
out) . 

2. Attitude- -The best experience I have had in a good long 
time. We tend to get biased in our judgment of other areas of 
learning and the efforts of others to bring about educational 
change and experi -mentation in education. I thought I changed 
my thinking as a teacher for the better. I became more rational. 

I learned new ways of approaching problems. This is one of 
the few really worthwhile experiences that I have encountered in 
the field of education. I think the plan should be applied to 
all the subject matter we now teach. What to teach, How to 
teach it, When to teach it and Where to teach it. I am certain 
that inroads of this method into education could and would improve 
the vast sum we now spend for education that are wasted because 
we have employed the wrong approach (Aristoblean vs. Scientific 
Method) at the wrong time and in the wrong place. 

I worked a total of 50 hours --other than eating meals over 
a four day period, and I enjoy what I did so much that it carried 
over in discussion at mealtime. This is a rarity on any project. 

I was so greatly impressed with the caliber of leaderships, 
especially Joe Kirk, one of the most impressive, brilliant, 
driving, dilligent and considerate ler.ders one could ask for and 
work with. 

It was a highlight, a rewarding experience one experience not 
only useful to me, but useful to society in understanding, and 
advancement in scientific thought processes and development. 
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My thanks should really be expressed to all those who tol- 
erated my opinion who conceded to my vigorous stands and we 
enjoyed intellectual fellowships. My thanks are to be expressed 
to those who asked me to participate. I was rewarded far greater 
than the monetary considerations involved! 

1. The different personalities and different educational and 
experience background of our leaders were very desirable. 

a. They (leaders) could have been rotated after completion 
of each book. (More standardization) 

2. It took Aaron Jestice longer to get to the point of a topic 
than Joe Kirk. The give and take of these leaders was desirable 
and for the most part beneficial. 

3. Almost imperative was more time to read the material- -some 
of the work could have been done on PACE by teachers, and lay 
people after they had several days of experience from the leaders 
of PATTERN Experience is the most important factor in making 
critical evaluations. 

4. Please--! would appreciate a resume of this inf ormation- -but 
I would like more information on the P.I.A. outline form. An 
excellent problem solving approach. Most desirable and useful 

for the solution of subjective problems in the critical evalu- 
ation scheme. 

The PACE project could have been done very exactingly had 
one read one book, and then spent shorter periods of time- -2- 4 
hours just discussing 2 to 5 projects involved. To do real 
justice to the evaluation . time, is the critical point, how 
much available, and how effectively used. 
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Teacher No, 2 

"I have never really participated in an educational ex- 
perience that generated as much effort in the field of original 
thinking as this experience did. I think that the evaluation 
efforts did arrive at balloting criteria that have validity 
as applied to the over all evaluation. 

I think that the pressure of time was an adverse factor 
that worked against a thorough analysis. There were many 
collateral benefits such as new bridges of communications 
witliin District 241. An area of cooperation between District 
241 and the State Dept, of Education, and a reinforcement of 
the cooperation between District 241 and our local parochial 
school system. 

I benefited greatly by working with the acknowledged 
experts in the field. I think I have gained valuable insiglits 
into the logic of decision making. I also think that this 
machinery can be successfully adapted to classroom instruction 
in logical thinking by the device of simulation games. I will 
be very interested in any advancements along this particular 
line of endeavor. 

In short, it was a very satisfying educational experience 
and I thank you for the opportunity to participate.” 

Teacher No, 3 

"Very enlightening in regard to analysis of the Mobile 
Science Laboratory program. Many intangible facets brought 
into discussion for dissecting the operation, staffing, location, 
funding, research projects and significance of the total program 
impact upon the whole community. 
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The very frank and open dissertations were a great aid in 
evaluating the past performance, assimilating the present and 
predicting the future use of the Mobile Science Laboratory 
program. 

The extremely concentrated and lengthy sessions were a bit 
tiring. I am still digesting the knowledge acquired. 

Very good sessions, good for me, good for futher expansion 
of the program. 

All participants of high caliber and qualifications!!!" 

Teacher No, 4 

"In all the sessions were stimulating. They taught me a bit 
more concerning other peoples’ points of view. I now have a more 
tolerant attitude concerning science. The interaction of faculty 
members was particularly worthwhile. 

I felt at times that too much stress was placed on resolving 
certain differences which had little to do with interpretation of 
criteria- -perhaps the numbers could have stood as orginially 
entered. After a while it should have been possible for the 
group to have arrived at some agreements without direct leader- 
ship--this would have saved much time. The sense of humor of 
"our leader" was particularly helpful in keeping a certain 
"lightness" in the group sessions - -much more serious drive would 
have hindered progress actually. 

I , myself, would have liked a bit more conciseness as to 
meals, arrangements ahead of time. It was very hard to plan too 
far ahead as a result." 
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Teacher No, S 

**I have worksd on all levels 1 1 felt that the entire process 
was very informative and interesting. 

The spirit was contagious at all levels. The students re- 
sponded enthusuastically and performed at a high productive 
level. They demonstrated a high level of understanding and ded- 
ication. People who had little or no knowledge of the Mobile 
Science Laboratory was able to see how it worked and were 
enthusiastic and worked extremely well as a team. 

I got a lot of insights in the study of the project reports. 
It brought understanding to strong and weakness that will help in 
doing next years program. Also where to aid the student to bridge 
gap between writing up results and field work. 

I felt that we were crowded for time and toward the end on 
very important work was necessarily not as well thought out and 
done too superficial." 

Teacher No, 6 

"At the outset let me say that the setting at the INN TOVVNE 
was very pleasant and conducive to the task of evaluating the 
Mobile Science Laboratory. It seemed to me the working relation- 
ship of the Evaluation Committee and the ALJ staff was excellent. 

I was indeed pleased to be of service to the Mobile Science Lab. 

The following comments cover the entire series of meetings 
but are more specific to the final four days, January 16-19. 

The schedule was altogether too tight, in that the hours 
were too long and too much to cover. Perhaps there would have 
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more time for reflective decisions, although I’m not certain 
there would be much change in the decisions. 

I would have appreciated having the projects in numerical 
order and the pages numbered for more ease in reference.” 

Teaaher No, 7 

"I felt the experience with the evaluation committee was 
quite unique. I believe the site (INN TOWNE MOTEL) added much 
to the inducement of good rational thinking. 

The lack of distractions and interruptions added much to 
the effort of concentration. The atmosphere of debate and 
discussion for balloting was unique for arriving at decisions. 

I feel that the awareness of this type of evaluation alone 
is quite stimulating. I did feel however, that we were trying 
to do too much in too short of a time, although I realize why 
this was necessary." 

Teacher No, 8 

"I felt the process to have value and possibilities in 
many areas of education evaluation. 

Time is a consideration both lack of and the degree to which 
we had to push. There is a degree of sensitivity training and 
taking a hard look at rather subjective material which I find 
desirable . 

In talking with others involved, I would think the small 
group makeup and the different leadership might have an effect 
upon thinking, output, and organization. Both positive and 
negative . 
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Again, I found the experience to be personally rewarding 
and I believe it was for most concerned. I will be interested 
in the type of results we receive from this type of input as 
I would like to be able to use the application in a variety of 
areas . 

I felt many times that the knowledge of the ALJ monitor 
could have had an influence on the group decision. (Especially 
project level and up). Time did not permit settlement of this 
item. " 

Teaaher No, 9 

"The evaluation program was a most interesting and unusual 
experience. I’m certain that it broadened our ideas about 
evaluation and should stimulate improvement in our own eval- 
uation opportunities. 

Perhaps the number of summer science projects to be eval- 
uated could have been reduced, or else one more committee to 
study them could have been used (increased in cost?)." 

Teaaher No, 10 

"I found the interaction within the evaluating sessions very 
stimulating and most rewarding experiences of the weekend to me 
as an educator. I feel, however, that less should have been 
demanded of the group in the way of amount of material to be 
evaluated. I felt less time should have been spent on evaluation 
of student projects and more on the evaluation of the Mobile 
Science Laboratory itself." 

Teaaher No, 11 



"My role with the evaluation committee was quite limited. 




I took part only due to the illness of another member. I hope 
that my small contribution was in some way meaningful and con- 
veyed to others the effectiveness of the Mobile Laboratories 
in the elementary science program. 

I was impressed with the dedication and energy displayed 
by all members who worked the four days of January 16-19. It 
was their objective to arrive at a constructive and accurate 
evaluation . ” 
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4. Students Participating on Balloting Sessions 



r. 



a. Name, Grade, School List 



n 

0 

n 

0 

u 

0 



0 

u 



Name 


Grade 


School 


Anderson, Janet 


8 


Brookside 


Anfinson, Julie 


8 


Southwes t 


Ash, Charles 


9 


Southwes t 


Botton, Susan 


9 


Brookside 


Boyum, Nancy 


8 


Brookside 


Bruce, Jean 


9 


Brookside 


Colby, Raymond 


9 


Southwest 


Colby, Leigh 


10 


Sr. High 


Dahl, Marcia 


9 


Brookside 


Denton, Douglas 


9 


Southwest 


Dreisbach, Nancy 


8 


Southwes t 


Durgin, Mike 


9 


Southwest 


Erickson, Mark 


11 


Sr. High 


Erickson, Mike 


8 


Southwes t 


Gendler, Stacey 


9 


Brookside 


Gregarson, Jonathon 


10 


Sr. High 


Gurwell, Joan 


9 


Brookside 


Halverson, Dan 


9 


Southwes t 


Halverson, Paul 


8 


Brookside 


Hanson, Kurt 


9 


Southwest 


Harrison, Sue 


8 


Brookside 


Hrinek, Andy 


9 


Brookside 


Hromadko, Gail 


8 


Southwes t 


Hromadko, Gary 


11 


Sr. High 


Jensen, Alan 


11 


Sr. High 


Jensen, Annette 


11 


Emmons , Min 
High School 


Jensen, David 


9 


Brooks ide 
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Johns rud, Lorraine 


11 


Sr. High 


Johns rud, Marlene 


12 


Sr. High 


Lair, Patrick 


8 


Brookside 


Langerud, Cindy 


9 


Brooks ide 


Lubitz, Marvel 


9 


Brookside 


Marsinski, Bruce 


10 


Sr. High 


Matthies , Emily 


9 


Brookside 


Moe , Bradley 


9 


Brookside 


Modderman, Jane 


8 


Southwes t 


Parrish, Sharon 


10 


Sr. High 


Phillips, Ann 


8 


Brooks ide 


Phillips, William 


10 


Sr. High 


Quackenbush, Jana 


8 


Southwes t 


Roberts , Mary Jane 


11 


Sr. High 


Roberts , Warren 


sophomore 


Macalaster College 


Saunders , Mary 


9 


Brooksi de 


Schoeppach, Jean 


8 


Southwes t 


VanRiper, Denise 


10 


Sr. High 


Vandergrift, Judy 


10 


Sr. High 


Vaughn, Charles 


11 


Sr. High 


Vaughn, Robert 


10 


Sr. High 


Wahls trom, Darrel 


10 


Sr. High 


Wedge, Vicki 


10 


Sr. High 


Willi ams , Deb r a 


8 


Brookside 


Ziegler, Mary 


8 


Brookside 


Zgoda, Terri 


8 


Brookside 



A-17 



b. Student Evaluation Comments 

student No. 1 

I think the Planning should be done before going out on the 
project. This would leave more time available in the field. The 
PIA method is good and quite helpful, but I think it gets too in- 
volved. 

The definitions may and are needed at times, but defining 
things such as loud, lake, and such is useless to me. The data 
format and planning of it is very helpful in sticking on what 
you goal is. The implementation should consist of, in my opinion, 
data, schedule kept and other things you have done. By writing it 
down you will keep in mind your objectives for the project. The 
plans for evaluation should have been explained in part so that 
these cards would have contained all the information obtained 
plus the conclusions and future plans. It is not the system it- 
self, but the extensive detail that bothers me. I think the 
final results and "collection and development of sufficient data" 
should be stressed more than the planning after you've started. 

The evaluation is terrific! It was very helpful. I think 
everyone should get to see just a little bit how it works. One 
big problem in my mind is the fact that the information found was 
not necessarily included. Perhaps the information was listed v\rith 
the specimens, but we were not told what the correction sheets we 
got were for, so this could be corrected. Another problem was the 
fact that several people evaluated their own projects. It was hard 
enough knowing that this person did more but it doesn't show it, 
much less trying to forget what you yourself did or, for that 
matter, didn't do. Even still, the comments made about your 
project were very helpful. 
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It was liard to decide but being in groups and discussing it 
made it easier. The way it was done seemed wrong and hard to 
understand at first, but putting them together, I could see how 
it makes a complete picture. Something that was hard to compare 
was such different projects, yet if they were very similar, it 
would be hard also. This technique is fascinating and I am 
looking forward to seeing what is gotten out of it. 

Student No. 2 

The unilateral use of standardized cards, though beneficial 
as it may be to the analysis of various learning methods, is, in 
itself, a deterrent to the intent of the Mobile Science Lab 
project-this intent being the self instruction of students in 
scientific method with assistance by competent instructors. Form- 
ulation of one’s own procedure, schedule, data format, etc., is 
essential to the success of one's project; however, time spent 
in repetitious recording of various phases of one’s project limits 
the actual time available for execution of the phases and impairs 
the quality of the investigation. Instruction in writing papers 
and planning the essence of a project before the field trip would 
be more worthwhile and more time saving than directions for 
filling out cards can possibly be. Admittedly this would result 
in a less uniform project procedure when applied to a number of 
students, but the question arises --who is supposed to benefit 
from the experience of the field and classroom learning situations. 

Obviously the student. For this reason an effort should be made 
to eliminate any inconvience in project procedure for which the 
program is responsible. 

Enough can’t be said for the Mobile Lab porgram. An objec- 
tive look would reveal one of the best possible ways to have 
students learn about the many natural principles involved in their 
particular fields of study* There can be no doubt that direct 
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observation and analysis is of far greater benefit than this 
same observation and analysis related in a textbook. 

If it is so necessary to have a method to analyze the 
learning processes of students, I see no reason why the 
principles employed in the cards couldn’t be applied to a 
regular research write-up, thereby removing the redundancy 
now inherent in the cards, and providing for much more efficient 
science projects. 

Student lio, 3 

I found it difficult to evaluate a couple of projects because 
they were lacking in certain areas in their information. But it 
was easy with projects which had the information. 

I thought it was interesting to evaluate these projects. 

I thought it was well organized plan to evaluate these projects. 

The cards seemed like just extra work but they were really 
helpful once you got them. 

I hope we are able to continue with our program. It was an 
interesting and educational summer. 

Student No. 4 

How I would improve rating system? I’d have all the students 
exchange papers and then evaluate only one. Not very good. I 
didn’t think it would be so hard. 

How would I change write-up of report? I wouldn’t change the 
cards because that is a good way to put them in order if you mix 
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them up . 

I think this is a good system to work with because it is 
kind of an advanced way of helping a person understand his or 
her project better and also help a person reading this project 
to get more out of it. 

Science Program:! don’t think we should have to work on 
our cards as much because we spend over half our time working 
on them and usually the rest of the time we were always being 
taken some place. But still I like the cards since they help 
plan the project real well you ought to give us the cards at 
least a month ahead so we have time to plan and then that 
would give us more time to work on our project. 

I would like to see you have a science program last for 
about six weeks that way you could see more things. You 
could divide the six weeks into three equal parts, the first 
2 weeks work on the cards the second two weeks work on just 
the project and the last two weeks then take us around to 
see the things like sights, people (speakers), mines, etc. 

You should also have the Mobile Science lab start a science 
club that will be like have the meetings after school, take 
trips, camp out on holidays, etc. 

Student No, 5 

The PIA level evaluation was good in setup, even though you 
couldn’t tell if the project was good or bad, since they do this 
on project level, it’s okay. Some of the criteria were hard to 
answer, like whether it was flexible because sometimes you didn’t 
know if they changed their project or not. Also the question on 
resources since sometimes they might have used them and not 



written down the names of them. 



On the level of the whole project, I feel that the cards or 
whatever used should be started and nearly completed before 
starting anywhere, since they are time consuming. Also, I feel 
that a smaller number of phases be made because I think the 
teahcers need more rest than was given and also an opportunity 
to get away from the kids and each other. 

I feel that cards of the like are a good idea because you 

have to plan in detail before you start and you should know and 
understand what you’re doing. 

Evaluation by the students from the project is good because 
they know what the conditions were for the projects and have 
probably experienced them also sometimes too, its a good idea 
to put the students into groups with projects unlike their own 

so they can see if it is really organized and understandable 

because they can’t read anything into them because he doens’t 
know as much about the subject. 

I think that the people evaluating should be informed on 
what is happening in the evaluation because confusion of what 
to do in those handling or leading the evaluation of the group 
doesn’t help the ones doing the evaluation. 

Make sure a definite understanding is reached in the meanings 
of the criteria. 

Student No, 6 

1. More teachers to specialize in each area. So that almost 
every group of students has a teacher which knows about their area 
of study. 
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2. Also more help on using these cards because they’re very 
hard to understand. 

3. I tliink evaluation should consist of talking about the 
S.S.S. program. So if it is continued (hopefully) it is for 
those who learn and enjoy it. 

A. Use of cards: I feel that the cards serve mixed purposes. 
While on one hand they give a semblence of order, they also are 
extremely time consuming. I think that use of the cards should 
be a matter of personal preference. 

B. Length of programiThe program should be lengthened to 
encompass a fourth phase, (excluding basic). If this were done, 
the mobile unit could give programs to all students from grade 
seven to graduation. I think that upon completing all four phases, 
students should be entitled to high school extra credits in science. 

The amount of students assigned to each phase should be 
limited to no more than twenty five. 

C. Criteria (PIA):I think the use of planning, implementation, 
and analysis should become a matter of the students own needs. If 
the project by its nature, is not of the type that required careful 
analysis, for example, then that phase of tlie project should be 
omitted. 



D. A few suggestions as to the summer science program: (1) hire 
a full time director, (2) expand the program, (3) more classroom 
work, (4) more time to work on projects, (5) more detailed equipment 
(6) three teachers to a phase, (7) participation in other districts, 
and (8) besides biology, geology, and conservation, more subjects. 
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Student 7 

The cards system worked good this last summer. It helped 
the people to plan out their projects and to summarize it, I 
think we should continue with the courses and to spread to other 
cities , 

The summer program was interesting and fun, I hope there 
is a phase 4 or 5 the next year, I like the cards since it 
helped me to a better and well done project. It makes you feel 
rewarded when you take up a course like this and also shows you 
have accomplished something, I think you should give out rewards 
for joining it. If you could I think you should have it for 
longer than 4 weeks, 

I don't have anything to critize about it. 

Student No, 8 

"Information" might be substituted for "data", "Guidelines" 
could be used for "criteria", "Carrying out" could be used for 
"implementation" , 

Improvements : (1) Standardize methods for filling out cards, 
(2) Larger--more space on cards, (3,) A definite place to enter 
data from the field, and (4,) Simplified numbering system on 
cards , 

Student No, 9 

I feel this way is almost nearly perfect, except for one 
thing which is number "educational value for future endeavors". 
You should have stated more clearly what you wanted. Like how 
much they did learn and how important it is or how important 
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will it be in the future for the occupation you choose. 

The cards were a little difficult. 

Student lio, 10 

What diff iculuties were found in the evaluation processes? 
Judging each project separately was hard and difficult. 

Was it interesting or of value to you? Very much, it helps 
your future. 

Suggestions for changes in cards :More like scientific studies 
like term papers. 

General Comments: (1) Better food diet, (2) More leisure time, 
(3) Better recreation facilities, (4) Hot water, and (5) Better 
camping areas. 

Student No, 11 

1. I found it hard to do when people left out information, 
or when there was too much, and things that did not matter. 

2. I liked it, I got to see other peoples’ projects, and 
what they did, and how they did it. 

3. Have it so that there is one for each box instead of all 
of them add up to one. 

4. Instead of cards, I think a sheet of paper, and up on the 
top have plan and so on. 




5. You could feed us better. 
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Student No, 12 

Tlie card idea wasn’t too bad. The only reason most of the 
kids hated it so was because with them they would have to do a 
little work. I did think, however, that the cards could have 
been better explained so we would know how to use them. 

I was disappointed in the field trips themselves. The time 
we could have used for going to fossil areas was all taken up by 
specially planned lectures and tours which hardly had anything to 
do with many of the projects, were boring, and were re-runs from 
the Basic program. As a result my project was a complete flop. 

I collected no fossils. 

Student No, 13 

Mow would I improve the rating system?:! would have the 
rating equal out to a bigger number but I thought that it wasn’t 
so hard after you got use to how it worked out. 

How would I change the write-up of project?:! would have 
the cards in an order in which they would follow in the project 
you are working. 

Student No, 14 

1. ! would leave it the same, as this is the best method to 
evaluate a program of this size. ! like it personally, and ! 
enjoyed it if it wasn’t so long. 

! would leave it also, but ! wouldn’t put as great an emphasis 
on tlie cards used to write it up. 
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Student No, 15 

I think you should have more time to collect and organize 
your specimens for your project. The cards take up all the time 
so you don’t have time to get the specimens and materials you 
need for the project. 

I was on Phase I this year with the cards and on Basic last 
year without them, then I got a lot more done on basic and got a 
good fossil collectibn, but this year I didn’t get anything the 
reason is because I spent so much time on the cards. 

Student No, 16 

f 

I 

^ 1. That they should have more teachers to teach summer science 
and so that when students need help the teachers could help. 

2. Have more help on cards or not as many cards. 

Student No, . 17 

1. The cards were very difficult because I don’t think we 
really got a complete explanation of the cards. 

2. This group is a perfect size. This job was interesting. 
Student No, 18 

I think they could be arranged a little bit better, but T 
think they helped a lot. 

Student No, 19 

I think that the cards are the things that brought down 
the enjoyment of the program. 




student Ho. 20 



If we over evaluate again, I think they should have numbers 
on the pages of the group book so we could find things quicker. 
But, personally, I don’t think we should use this type of cards 
again, because I don’t believe that they are that understandable, 
especially for kids in the 7th grade, and I don’t think they are 
necessary. I think it would be just as easy to write a papei on 
your project instead of doing cards. I think they take too much 
time and if we did do it again they should make the cards more 
understandable and easier. 

Student Ho. 21 

For ease of evaluation the project numbers should be pre- 
sented in ascending order in the booklet and the pages should 
be numbered. In addition the listing of projects on the eval- 
uation sheet should be in the same order as appears in the 
booklet . 



The terms in the criteria section were not chosen with 
clarity in mind and words such as expedient should not be used. 

If at all possible a project whose end product is a map or 
chart (e.g. surveying) should have this chart included. 

Student Ho. 22 

It works quite well, though I would appreciate more time for 
the evaluation- - (spread out more). 

I would have more written about the data gathered and more on 
what they used and how they did it. 
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Student Eo, 23 

You should put the reports in the book in order. I think 
it would save a lot of time instead of trying to find the correct 
report. I think you should use the terms so you can understand 
it better. I think the evaluation was good and you could under- 
stand it. 

Student No. 24 

To put it in a different way so it would be easier to fill 
out the cards . 

Student No. 25 

1. I found it difficult in some cases to evaluate these 
proj ects . 

2. Yes, I thought it was interesting to work on the projects. 

3. No. 

4. I think we should either keep them the way they are or 
ban them. 

5. None. 

Student No. 26 

1. (a) It helped my note taking, and I have a good basic 
understanding of the evaluation. 




(b) At first I didn't understand it, but who did? 
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2. I judged the projects a little on my level, but with the 
PIA I did it strictly between the PIA itself. 

Student Uo, 27 

I understood the evaluation. I didn’t like the cards at all. 
They were hard to do, and didn’t make much sense. 

To change this course, I would leave out the cards, I don’t 
know what I would put it its place. 

I feel this is a fair evaluation, but I don’t think a person 
should evaluate his own project. 

Student No» 28 

1. Yes, I understand this form of evaluation. I didn’t really 
like it very much because it got boring at times. 

2. I think they should leave the cards and the form the way 
it is now because you can get more out of it this way. 

3. I was judging on a same level as my own work. 

4. I do feel that this is a fair evaluation of projects. 
Student No» 29 

1. Yes, I understood this form of evaluation. I really didn’t 
like all the work involved. 

2. If I could change it I would make the cards shorter. 

3. Yes, I mostly judged them on the level I was in. 

4. Yes, I think this is a fair evaluation. Yes, I will accept 
the judgment of the group on my project. 
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B. Relevance Guide Book 

1. Introduction 

It has been indicated that no analysis can be better than 
the expertise of the evaluators and the information available 
to them. As much effort as practical was devoted to this task. 
This section discusses the questionnaires and an analysis of 
the data that was provided to the educators, includes a repre- 
sentative program description that was one of ten prepared from 
a review of over 200 program proposals, and the data received 
from responses to individualized letters that were mailed to 
the included list of 134 PACE Program Directors. 

In addition, ALJ Associates, Inc. collected a variety of 
source and background material to foi^ the basis for the rele- 
vance network and relevance guide book. Our research was 
directed toward collection of relevant PACE educational data 
and was comprised of a series of visits, telephone calls, and 
letters to various educational authorities and directrates. 
Included among these contacts were the Office of Education, the 
National Education Association, the Library of Congress, the 
Educational Service Bureau and four of the Universities in the 
Washington, D. C. area. 

A briefing to the Office of Education on the evaluation 
was conducted in December 1968 by the staff of ALJ Associates, 
Inc. Also, a series of meetings were conducted in Albert Lea 
informing the evaluation committee of progress in structuring 
the network and developing the criteria. The committee was 
composed of several teachers, principals, and the state science 
advisor. The first meeting was held in early November, fol- 
lowed by an early December meeting on questionnaires, a late 
December session for students to assign relevance, and a mid- 
January session for committee relevance assignment. 
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The final sessions were tape recorded to enable careful 
examination of the discussion periods. This allowed a detailed 
analysis of the issues enunciated during the meeting. 

2. Questionnaires 

a. Development 



The questionnaires were developed specifically to form the 
basis for the Relevance Guide Book at the Function and Curriculum 
level. The need for information beyond that immediately available 
to the relevance assigners was evident. To inform questionnaire 
recipients about the Mobile Science Laboratory program a series 
of newsletters was published on its various facets. These were 
then distributed through the schools to parents. 

One newsletter was specifically devoted to the elementary 
program, while the rest covered various parts of the secondary 
program. The newsletters were composed at Albert Lea and 
printed by ALJ Associates, Inc. (Samples of elementary and 
secondary newsletter are included.) 

•< 

The questionnaires developed through a joint effort of the 
educator evaluation committee and ALJ Associates, Inc. Each 
group developed a set of questions, which was then merged by 
ALJ Associates, Inc. The merged set was reviewed by the eval- 
uation committee, amended and printed by ALJ Associates, Inc. 

The questionnaires did not contain the normal set of survey 
yes/no questions. Instead most questions were directed at 
amplifying a facet of one of the criteria. Some reference 
questions were also included. A separate questionnaire was 
composed for each of the groups of interestischool administrators, 
counselors, educators, secondary students, and parents of secondary 
and elementary parents. The last two groups could not be polled 
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by the same kind of questions as the first groups, but an effort 
was made in all the questionnaires to force a choice by eliminating 
middle-ground and "don’t know" answers. 

The questionnaires were sent to all school principals , 
counselors and teachers in the school district. All secondary 
students who had participated in the program also received 
questionnaires. Because of time constraints, a random sample of 
300 elementary and 300 secondary parents were selected to receive 
questionnaires . 

Problems with air freight and the weather delayed receipt 
of the questionnaires in Albert Lea and their return to Washington 
for processing. However, this problem was overcome and processing 
completed in time to publish the Relevance Guide Book. 

Each question of each questionnaire was analyzed, summarizing 
key points. Then all questions which applied to a criterion were 
merged to produce the final text under the criterion. The following 
is an example: 

Secondary Parents 

Question 1 

The parents of secondary students felt that the Mobile 
Science Laboratory affected their children in several significant 
ways . 



83% of the responding parents felt that the Mobile Science 
Laboratory engendered increased respect for nature and natural 
science. This selection was by far the most often chosen to 
describe the impact on their child. 
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Next most popular response was that of increased self- 
improvement and social maturity. 65% of the parents felt that 
this was an important feature of the Mobile Science Laboratory 
experience . 

58% of the parents felt that participation in the Mobile 
Science Laboratory influenced the grades of the children. 51% 
felt the program fulfilled recreational needs. 

44% felt that hobbies and out-of-school activities were 
influenced by the Mobile Science Laboratory. 27% felt that 
selection of science courses at school had been effected while 
19% felt future jobs or profession had been influenced. 

13% of the parents added a variety of comments ranging 
from no influence to filled vacation time to stimulated 
curiosity about science. 

The final text consisted of merging similar analyses from 
ten other questions. 

The following table refers the questions to the criteria. 



Network Level Criteria Questionnaire # 



Function 


26 


Ad 

EP 




27 


Ad 




28 


Ad 




29 


Ed 


Curriculum 


30 


Co 




31 


Co 




32 


Co 




33 


Co 



1,2,5; Co 1; Ed 4,7; Sp 1,2; 

1,2,3; SS 4 

3,4; Ed 2,3,4 

3,4; Ed 2,3 

5 

3; Ed 8; SP 1,3,4; EP 1,2 
1,3; Ed 1,4; Sp 1,3,4; EP 1,2; SS 
3; Ed 9; SP 1,3,4,; EP 1,2 
3,4; Ed 6, 10; SP 1,3,4; EP 1,2 
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Ad=Aministrator 
Co=Counselor 
Ed^Educator 
SP=Secondary Parent 
EP=Elementary Parent 
SS^Secondary Student 

A copy of the questionnaires and of the Function and Cur- 
riculum Relevance Guide Book follow. This guide book is in- 
cluded as it provides a good insight of the reaction of the 
District 241 community to the MSL program. 
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b. Sample Questionnaires 

MOBILE SCIENCE LABORATORY EVALUATION 

Check one or more of the following indicating your association 
with the Mobile Science Laboratory program. 

A. Participant in one or more of the courses offered. 

B. A son or daughter has participated in the Mobile Science Labor- 
atory program. 

C. Have seen the Mobile Science Laboratory in operation on the 
elementary school level. 

D. Viewed film on Mobile Science Laboratory. 

E. Inspected Mobile Science Laboratory as part of open house or 
PTA meeting. 

F. Read or heard of program in: 

1. Newspaper 

2. Brochures prepared through school 

3. Other (Radio, TV, etc.) 

4. As a conservation volunteer 

G. Had personal contact with program or persons involved in planning 
the program. 

H. Teachers 

Science 

Other 

Mobile Science Laboratory Yes No 

TO ENSURE YOUR IDEAS ARE INCLUDED IN THE MOBILE SCIENCE LABORATORY 
EVALUATION, THE ATTACHED QUESTIONNAIRE MUST BE RETURNED BY MAIL 
BEFORE THURSDAY, DECEMBER 26. 
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ADMINISTRATION 

1) What possibilities could you offer to motivate teachers to 

participate in an in-service Mobile Science Laboratory program? 



2) In adapting the Mobile Science Laboratory program to Adult 
Education rank numerically the following potential benefits: 

a) General science background 

b) High School science credit 

c) Avocation aspect 

d) 

e) 

3) How can the usage of the Mobile Science Laboratory be changed 

to increase availability of space ^ materials and equipment » 
either where none existed or in improving present capability? 



4) If the Mobile Science Laboratory program were expanded , what 
impact would this have on availability of space, materials and 
equipment? 



5) How could the Mobile Science Laboratory be used to better serve 
as a public relations instrument for the school district? 
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COUNSELORS 

1) How has the Mobile Science Laboratory program affected requests 
for new course electives by the students? 



2) How does the Mobile Science Laboratory program fit into total 
academic course offering? 



3) From your experience, what benefits do students gain through 
participation in the Mobile Science Laboratory program? 



4) In considering the Mobile Science Laboratory program for student 
academic recognition, rank the following: 

a) High school science credit 

b) Achievement award, e.g. certificate, microscope 

c) Augment science grade 

d) 

e) 



5) What kind of Mobile Science Laboratory information is needed to 
advise the student? 
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EDUCATOR 

1) Rank numerically the following educational student benefits 
derived from the Mobile Science Laboratory program: 

a) Independent self-study 

b) Extension of previous learning 

c) Real-world applicability 

d) Laboratory experience 

e) Scientific awareness 

£) 

g) 

2) How can the usage of the Mobile Science Laboratory be changed 
to increase availability of space, materials and equipment, 
either where none existed or in imporving present capability? 



3) If the Mobile Science Laboratory program were expanded , what 
impact would this have on availability of space, materials and 
equipment? 



4) What impact would discontinuance of the Mobile Science Laboratory 
have on the student's learning science? 



5) To improve your science teaching, what additional training would 
you be interested in acquiring through the use of the Mobile 
Science Laboratory? 
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6) Rank numerically the following teacher benefits in considering 
field training in the Mobile Science Laboratory in the summer: 

a) College credit 

b) Financial compensation 

c) Fulfillment of school contractual agreement 

on extended school year 

d) Developing and extending science curriculum studies 

e) 



7) If the Mobile Science Laboratory program were expanded, what 
impact would this have on augmenting student familiarity with 
science? 

8) Rank numerically the following student physical benefits derived 
from the Mobile Science Laboratory program: 

a) Exposure to natural environment, e.g. fresh air 

b) Organized activity, e.g. sports 

c) Individual activity, e.g. rock collecting 

d) 

9) Rank numerically the following student social benefits derived 
from the Mobile Science Laboratory program: 

a) Increased group relatedness 

b) Contact with government and community resources 

c) Camping experience 

d) 

10) Rank numerically the following indices of student self - improvment 
derived from the Mobile Science Laboratory program: 

a) Motivation 

. b) Leadership 

c) Personality 

d) Self-reliance 

e) 
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TO THE PARENTS OF SECONDARY STUDENTS 



1) In what mannor has the Mobile Science Laboratory program affected 
your child? (Mark as many as applicable) 

a) Increased respect for nature or natural science 

b) Selection of science courses at school 

c) Success at school (grades, new friends, etc.) 

d) Hobbies, out-of-school activities, scouting, etc. 

e) Future job or profession in science area 
f) Self-improvement, social maturity 

g) Fulfilled recreational needs 
h) 

2) What is your opinion of the scope of the secondary Mobile 
Science Laboratory program? 

a) Continue program at present level 

b) Expand the program 

c) Discontinue the program 

d) Not familiar with the program 

e) 

Please explain your response: 



3) What Mobile Science Laboratory science activities has your 
child conducted at home? 

4) What Mobile Science Laboratory activities in particular has 
your child discussed iwth you? 



5) Would you be willing to participate in local financing of the 

Mobile science summer science program to ensure its continuance? 

a) No 

b) Yes As taxes, As nominal individual partial 



payment 
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TO THE PARENTS OF ELEMENTARY PUPILS 

1) What Mobile Science Laboratory activities in particular has 
your child discussed with you? 



2) What Mobile Science Laboratory science activities has your 
child experimented with at home? 



3) Do you want your child to take part in the junior high summer 
Mobile Science Laboratory program? 

a) Yes 

b) No 

Please explain your response: 



4) What is your opinion about the elementary Mobile Science 
Laboratory program? 

a) Continue the program at present level 

b) Expand the program 

c) Discontinue the program 

d) Not familiar with the program 

e) 

Please explain your response: 



SECONDARY STUDENTS 



1) How does the Mobile Science Laboratory program fit into your 
academic course plans? 



2) In which of the following does the Mobile Science Laboratory 
program affect you? 

a) Future plans: 1. High School 

2. College 

3. Occupation (including military) 
b) Developing outside interests (Church activities, 

hobbies, 4-H, scouting, etc.) 
c) Self (improvement, motivation, reliance, leader- 
ship, understanding others) 

d) School achievement (awards, grades, etc.) 

e) Other 

f) No help 

I 

3) What offerings or improvements could be made to Mobile Science 
Laboratory program? 

a) One week excursion 

b) 6 -week program 

c) High school science credit 

d) Achievement award, e.g. certificate, microscope 

e) Augment science grade 

f) Others 



4) What experience and knowledge have you gained through partic- 
ipation in the Mobile Science Laboratory program? 
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5) What was the main reason you participated in the Mobile Science 
Laboratory program? 

a) Interest 

b) Parents insisted 

c) Seemed like nice vacation 

d) Other 

6) How many years have you been in the Mobile Science Laboratory 
program? 



a) Number of years in program 

b) Grade in school 
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C. Analysis of Quesdonnaices 
a. In trod uc don 

This analysis presents data gathered from questionnaires 
sent out to secondary students, parents of both secondary and 
elementary students, school counselors, educators, and school 
administrators . 

The purpose of this Guide Book is to present some of the data 
necessary to assign authoritative relevance numbers using each of 
the criteria. The data here was collected through use of questions 
which were aimed at a particular criterion. The content and qual- 
ity of the answers explain the variation in detail accorded to 
each criterion. Where possible the data was presented as it 
applied to each element at the level. 



f 
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RELEVANCE GUIDE BOOK 
FUNCTION AND CURRICULUM LEVELS 



PREPARED FOR: 
EVALUATION OF THE 



MOBILE SCIENCE LABORATORY 



26. PARTICIPANTS 



To what degree does participation in the MSL program augment 
participant curiousity , knowledge and familiarity with science 
through elementary use^ secondary use, teacher use, and com- 
munity use? 

This criterion assesses the impact of the MSL program on the 
participant, as the program is used by the element. The impact 
we are measuring involves the importance to the participant of 
the experience for educational benefit now and in the future. 

Elementary Use 

Elementary participation .takes the form of scientific work, 
organized into units which are performed by each class in the 
MSL. The units are directed at a particular grade level and 
involve basic concepts in science such as magnetism, sound, 
light, rocks, etc. The classes are brought into the MSL (1/2 
the class at a time for large classes) spending an amount of 
time which varies according to unit difficulty and grade level. 
This allows exposure to the child of a science program that his 
teacher might not be able to provide due to the lack of an ad- 
equate science background, as is often the case for the general- 
ist that the elementary teacher must be. Expanded plans would 
include field trips of short duration during the school term 
or summer. 

Teachers find the students more interested in science and 
better prepared to work in laboratory situations. Many teachers 
feel that the student, after participating in the MSL program, 
demonstrates a better understanding of science and that the 
opportunity to use the MSL provides an important motivative 
device for accomplishment in science learning. Some teachers 
find that there is a carry-over of this interest in learning to 



other subjects in the elementary class. Several teachers were 
unable to say what impact expansion of the MSL would have on 
augmenting elementary student familiarity with science, although 
most said that the MSL program as now constituted was an excel- 
lent program for their students. 

Elementary parents noted that their children very frequently 
discussed experiments that they had observed or performed in the 
MSL. Pupils mentioned equipment that they were familiar with 
from use in the MSL program. This equipment included instruments 
such as machines, pulleys, magnets, compasses, helium balloons 
and bunson buners. The children expressed new interests pro- 
moted through the MSL program such as collecting insects and 
rocks. Parent- child discussions involved specific experiments, 
skills using scientific equipment and new interests stimulated 
by using the MSL. 

Elementary parents stated that simple experiments were done 
at home as a result of the child’s participation in the MSL. 

These experiments used some of the following equipment iwheels , 
pulleys, magnets, chemicals, and thermometers. Another activity 
fostered by the MSL was rock, fossil, insect and crayfish col- 
lection. Observation techniques were carried on in the home as 
well. These were applied to a thermometer experiment, butterfly 
hibernation, insect studies, along with observation of the effects 
of air pressure and gravity. 

66% of the elementary parents stated that they want their 
children to participate in the junior high school MSL program. 

They justified this response with the following reasons : their 
children liked science, they enjoyed working in the MSL in the 
past, the MSL had provided good learning experience, the program 
offered the opportunity to travel, the MSL broadened the child’s 
understanding of science and nature. Parents also thought that 
the MSL program provided a valuable learning experience, as well 
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as broadening the child’s knowledge of his environment. One parent 
found that the elementary MSL program was of the most value be- 
cause "educators couldn’t mess it up like they did everything 
else . " 

The 34% of the parents who did not want their children to 
participate in the MSL in junior high felt that: their child had 
no interest in the subject, the children were too young to decide, 
the MSL interfered with other summer activities, the program caused 
too great an inconvenience for parents in that they would have to 
provide transportation, that they would be out of the area in the 
summer, or that rural children had too much work to do at home. 

Secondary Use 



Junior and senior high school participation takes the form of 
summer field excursions for extended periods to places of scientific 
interest around the state. The students plan their own study pro- 
ject following an outline prepared by the project director. They 
are free to select any study topic they wish. Selections include 
collecting fossils, mapping lakes, and studying human sensitivity 
with respect to natural environment. The program has taken place 
during the summer months and could be expanded to include weekends , 
vacation periods during the school year, or after-school programs. 

Counselors felt that participation in the MSL program had no 
apparent effect on student selection of electives. One noted that 
the junior high curriculum was prescribed and that little impact 
could therefore be noted. 

Teachers felt that the impact of the MSL was evident in 
increased student interest and understanding in science. Several 
noted that increased field experience and increased ability to 
function in a laboratory environment were evident. Some few 
simply noted that "there is no end to the benefits." Other 
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teachers saw an increased appreciation of the outdoors and of 
conservation, while some noted that there was more individual 
experimentation and that more self-discovery and self-directed 
activity was occurring. 

The parents of secondary students felt that participation in 
the MSL affected their children in several significant ways. 83% 
of the responding parents beleived that the MSL engendered increased 
respect for nature and natural science. 65% of the parents felt 
that they saw increased self-improvement and social maturity in 
their child. 58% felt that participation in the MSL influenced 
the grades of their children. 51% felt that the problem fulfilled 
recreational needs, while 44% felt that hobbies and out-of-school 
activities were influenced. 27% felt that the selection of science 
courses at shcool had been affected, while 19% felt that future 
jobs or professions had been influenced. 13% of the parents made 
various comments such as **the MSL had no influence,” "the MSL 
filled vacation time,” "the MSL stimulated curiosity about 
science . ” 

Many parents saw increased home scientific activity. Frequently 
mentioned were rock collecting and other collecting projects along 
with report writing or simply work on projects or experiments. An- 
other frequent response, however, was that no activity from the 
MSL carried over to the home. 

General discussions about the MSL program were carried on in 
the homes of the secondary students. The overwhelming response 
by parents was that the students discussed their projects and the 
field trips. Some mentioned that "day-to-day activities" or "all 
he participated in" were the principal topics. 

Secondary students felt that the MSL provided immediate aca- 
demic benefit in that it aided students in present science classes 
dealing with geology, biology, and paleontology, and that it 
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promoted math appreciation. The MSL program also provided a back- 
ground in the physical sciences, biology, and conservation in the 
opinion of the students. Use in the MSL of techniques of the 
scientific methodology broadened interest in science and served 
as a supplement to classroom learning. Students also felt that 
the MSL fit their academic science plans primarily in its appli- 
cation to future education plans. High school, college and future 
careers were about equally mentioned. While most reaction was 
that the MSL was academically valuable, 20% of the students were 
either undecided as to how it was valuable or were negative in 
their reaction. One child felt the MSL program interfered with 
band practice and on that basis was negative in his appraisal. 
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Students were asked to state what experience and knowledge 
they gained through participation in the MSL program. Generally 
most students listed specific skills and knowledge gained as 
pertained to their individual project. For example, the student 
whose project had to do with fossils learned most about how to 
identify and find fossils. Skills that were applicable beyond the 
MSL were also learned such as how to use reference materials, how 
to work in small and large groups for task accomplishment. Lis- 
tening skills were also improved as reference people and experts 
contributed to the individual students learning experience. Ob- 
servation skills were emphasized as scientists demonstrated ex- 
periments and techniques. Social interaction among the students, 
teachers and resource people allowed the students to learn to 
work with and get along with people. They learned to make new 
friends. Self-reliance and a sense of responsibility were 
mentioned by many students as a direct outgrowth of their work 
in the MSL program. 

Recreational skills in camping out were mentioned by several 
students, while others felt the MSL offered the opportunity to 
visit new places. Still others enjoyed the exposure to the 
various types of jobs that had to do with science. One student 
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learned to use a camera which excited a whole new curiosity for 
him. Broadened interests and expanded knowledge in various fields 
affected each of the cliildren taking part in the MSL. 



Teacher Use 

In-service teacher use of the MSL would strengthen the 
science program substantially in the elementary area where gen- 
eralists prevail. The MSL could be used as a research facility 
for junior and senior high science instructors who would have 
more specialized interests. It could be used as an adjunct 
laboratory especially for teachers, allowing some of the limi- 
tations now present on teacher research (such as space and avail- 
ability) to be alleviated. 



Community Use 

Community participation in the MSL is also possible. Admin- 
istrators felt that the emphasis of such a program would be most 
beneficial for general science background. High school credit 
for high school degree and the avocation improvement aspect were 
also mentioned as being relevant. The MSL could be moved around 
to the various shopping centers or other central locations for 
convenient access. Community participants would learn about 
scientific concepts that could be very useful to them in every 
day life. For example, fertilizer and nitrate experiments could 
be performed to help increase the understanding of their uses and 
impact on farming. Industrial chemical experiments might be 
possible. Also experiments and independent research would be 
possible for the adults who would desire it. The MSL would 
enable the community to keep abreast of the rapid scientific 
advances being made around the country today. It could demon- 
strate the impact of pollution through actual observation on 
extended field trips, as well as through laboratory experiment. 
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27. FACILITIES 

To what degree does the use of the MSL by the function level 
element increase the space j materials and equipment avalilable to 
i'nstruct in laboratory science? 

This criterion assesses the impact of the MSL program on the 
facilities which it makes available to the function element. It 
is simply the fact that more facilities are available, not that 
new facilities are offered where none previously existed. 

In the elementary schools there is, of course, no scientific 
laboratory available; the MSL therefore represents a great in- 
crease over that condition. The MSL could serve as a resource 
center in science at any school. Equipment could be loaned from 
the MSL to the elementary classroom to promote follow-up activ- 
ities. Use of the MSL also preserves classroom space. Many 
elementary teachers felt that this was one of the principal bene- 
fits of the MSL from the facilities standpoint. Laboratory 
experience will be available for the elementary student. 

In the secondary use, the MSL provides additional laboratory 
space and provides the opportunity for field laboratory experience. 
The students have the opportunity to use scientific equipment and 
apply tlie equipment to a real situation. Field experiments are 
now a possibility for a number of secondary students. The MSL 
could also be used during the school year as a special experiment 
station for individual projects that might not be possible in the 
classroom. The MSL would also make available a number of labora- 
tories to fill immediate needs of the secondary schools at a lower 
cost than the present classroom laboratories. 

In-service facilities for instructing in science would be 
enhanced by the mobility of the MSL. It would provide a special 
location for in-service training in laboratory science. Also, it 
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increases the materials available for this training. Teachers 
could have a laboratory to themselves is one possible use of 
the concept. 

Community use of the MSL would familiarize parents with the 
laboratory in which their children participate. The MSL could 
be moved around the city to various central locations to permit 
easy public access. School open-house could be expanded to 
include sessions in the lab. The MSL would represent a large 
increase in facilities available for adult science education. 
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28. FULFILL DISTRICT 241 SCIENTIFIC EDUCATION REQUIREMENTS 

To whci't degi*66 does the MSL gTOV'ide to the fuyiot'Loyi teveZ 
etefneYit new opportunity to instruct in ZcborcLtory science by 
providing facilities where none previously existed? To what 
degree is the science education requirements of the element 
enhanced? 

The MSL provides new capability for the elementary program. 
There are presently no laboratory facilities at the elementary 
level in the district. The MSL allows students to move away 
from the textbook approach to science toward the operational 
approach which is more interesting and instructive. The labora- 
tory makes available to the students the talents of various 
resource people from the community and selected agencies . The 
level of science education can be independent of the interest 
or the background of the elementary teacher. Elementary teachers 
feel that the amount and quality of science instruction is sig- 
nificantly increased by the MSL. An elementary summer program 
would also provide significant field experience in science to 
the mature elementary child. Elementary teachers believe that 
removal of this program would necessitate its replacement by an 
alternative means such as science resources at each school or 
expensive closed circuit TV. 

The use of the MSL by the secondary students provides new 
opportunity for laboratory experience in the field. Students are 
now able to become familiar with the problems involved in plan- 
ning and developing an experiment outside the classroom in more 
or less of a real-world situation. Week-end field trips and after 
school use of the MSL could expand the present capabilities of 
the program. 

Since the MSL has become a part of the curriculum, in-service 
training in the laboratory is necessary for the elementary teacher. 
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29. PROVIDE INCREASED OPPORTUNITY FOR NEW TEACHING DEVICES 

To what degree does use of the MSL by the function level 
element increase the opportunity to use new teaching devices 
such as discovery method, natural environment, and participant 
se lf~ direction? 

The students are removed from the familiar classroom sur- 
roundings to the MSL for the elementary use of the lab. The lab 
offers varied instruction which allows the child to experiment 
more freely than he could in the classroom. This is a new con- 
cept in the science education of the elementary child. Elementary 
teachers feel that this is an innovative tool and several would 
"hate to see it discontinued." 

For secondary use of the lab, the student is taken out to a 
field location and there he is able to perform an experiment that 
he organized and planned. The student has complete responsibility 
for gathering data and directing his energies toward the completion 
of the project in the allotted time. He is very much self-directed 
and learns principally through the discovery method. 

For in-service and teacher use of the lab, the MSL offers the 
opportunity to pursue individual projects in an experimental atmos- 
phere free from limitations of space and availability of materials. 
Courses if field geology, field biology, or field botany could be 
offered. Conservation and general training in the use of equip- 
ment would be available. 

For community use, the MSL offers numerous possibilities for 
adult education using the laboratory. Some of these, for example 
involve education in use of fertilizer and other farm- oriented 
applications using the newest scientific teaching techniques and 
the laboratory to emphasize these points. 
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The MSL has generated and also facilitated solving this in-service 
training requirement. The secondary school science teacher is 
generally a specialist in his field and has need for in-service 
training only in keeping abreast of new improvements in his 
field. The MSL could also satisfy this requirement. 

The MSL provides new facilities and materials to the com- 
munity. Presently little or no laboratory facilities are 
used in the adult education program. The MSL would provide 
these facilities and materials at convenient locations. 
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30. PHYSICAL BENEFITS 

Assess the 'impact of the "oavvous cuiPiP'iQuta on the par*ti cipant 
in the MSL in terms of physical bene fits such as exposure to nat- 
ural environment^ participation in organized sports^ and various 
forms of other physical activity , 

Teachers felt that one benefit of the program was that of 
organized, physical activity. They considered individual activity 
and exposure to natural environment about equal in impact on the 
students and felt that both were substantially more important to 
the student than the organized activity aspect. 

51% of the parents of secondary students felt that the 
recreational needs of their children were fulfilled through 
participation in the MSL. Also several parents reported that 
their child was more interested in the outdoors and in being 
outdoors. Many students continued their projects such as 
collecting and weather reporting after their experience in 
the MSL. 



Students participated in organized games such as baseball 
and tag for recreational activities while at the camp site. 
Also individual activities such as exploring and walking were 
performed. The students for the most part lived in tents 
and were fed from the "chuck wagon" trailer accompanying the 
MSL. 
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31. COGNITIVE BENEFITS 

Assess the impact of the various ourrioula on the participant 
in the MSL in terms of cognitive benefits such as independent self- 
directed study, extension of previous learning, real-world appli- 
cation, laboratory experience , individual attention. 

Counselors felt that they had not seen distinct evidence of 
any cognitive benefits in so far as student selection of courses 
and electives was concerned. One counselor felt that the students 
were able to learn more science in the most natural way, through 
what they see around them rather than in the classroom situation. 

Teachers felt that the important aspects of the criterion were 
laboratory experience, followed by scientific awareness, real-world 
applicability, extension of previous learning, and independent 
self-study, in that order. They felt that there was a substantial 
increase in the student's capability to use equipment applicable 
to an experiment and in experiment planning. 

83% of the parents responding noted that the MSL engendered 
increased respect for science and nature. Increased knowledge 
about laboratories and equipment was also noted. 

Students felt that the MSL was important in giving them field 
experience and in requiring self-direction planning of their 
activities. They saw immediate academic benefit in some of their 
classes and appreciated the broadened background in conservation 
and in the earth and life sciences. 
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32. SOCIAL BENEFITS 

Assess the ‘impact of the vavi ous QUi!*x*i’Quta on the papti cipant 
in the MSL in terms of social benefits such as group relatedness ^ 
contact with community officials ^ camping experience^ leadership , 

Counselors felt that the program increased the ability of the 
students to cooperate and to accept other students* ideas. 

Teachers felt that the most important aspect was that of 
increased group relatedness. Contact with governmental and 
community resource personnel was judged second, followed by the 
benefits of camping together. 

Parents felt that their children had made a number of new 
acquaintances and had come back from the experience more tolerant 
of group activities and group action. 
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33. AFFECTIVE BENEFITS 

Assess the impact of the various curricula on the participant 
in the MSL in terms of affective benefits such as self-reliance ^ 
scientific awareness ^ motivation. 

Counselors felt that some attitudes toward learning had 
changed under the impact of the MSL program. 

Teachers felt that motivation was the principal benefit 
generated by the program. Self-reliance, leadership ability, 
and personality changes followed in that order. 

65% of the parents felt that increased interest in self- 
improvement was evidenced after the program and the same percent 
saw more social maturity in their children. 

The students felt overwhelmingly that their attitudes toward 
self-improvement, motivation, self-reliance, and understanding 
others changed after their MSL experience. 
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3. Sample Program Write-Up 

a. Introduction 

The following is an example of data prepared for the Rele- 
vance Guide Book program level from the proposals, and project 
directors responses. 

b. FLOATING SCIENCE LABORATORY DESCRIPTION 

Abstract 

A floating marine science laboratory program will be 
offered to junior and senior high studnets from all public 
and nonpublic schools in the metropolitan county. The various 
disciplines of science- -Biology , Physics, and Chemistry- -will 
be integrated through the medium of oceanology to expand and 
enrich student understanding. Instructional materials will be 
developed for students of all ability levels. A 65-foot 
converted commercial sportfishing boat will serve as the 
laboratory. The boat will be equipped with equipment 
necessary to conduct scientific experiments. For example, 
students will set traps, collect and examine specimens 
through microscopes, and record data scientifically. In 
addition to learning about ocean plants and animals, the 
students will be instructed in navigation and in the physical 
properties of water, The students should gain an appreciation 
for the complexity of sea life and a realization that there 
are many vocational opportunities associated with marine 
science. Approximately 4,750 students will participate. 

Emphasis on an exemplary program. 

1. It is intended that the program of marine study shall be 
carried out as a floating laboratory boat. The design of the 
program and facilities are to be so structured as to fully meet 
the needs already described. The program shall utilize the 
techniques avaliable to commercial fishermen, scientists, and 
researchers to make soundings, collect specimens, and to accom- 
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plish many other related phases of marine study that are deemed 
desirable . 

2. A typical program in marine ecology might follow the following 
steps . 

a. Advance information of the expectations and goals of the 
floating laboratory so that the classroom teacher may 
prepare the students. 

b. Departure from dock aboard boat with an introduction to 
the day’s activities. 

c. Examination of an anchovy (Forage fish) by each student 
with special attention to physiological features. 

d. Circle previously set lines while the fathometer shows 
the topographical features beneath the sea. 

e. Pull set lines and put catch on to sorting table. 

f. Indicate difference in species. 

g. Have students sort species and examine internal organs 
and stomach contents. 

h. Circle previously set gill nets with fathometer to show 
habitat of surface feeding fish. 

i. Follow same procedure as used with set lines. 

j. Circle previously set crustacean traps and follow above 
procedure . 

k. Follow-up study program in classroom upon return for 
evaluation of findings and relationship to marine study. 



3. With several sets of nets, traps, and setlines a good 
representative sampling of the inshore animal life could be 
assured and sample specimens for future study could be secured 
for the individual schools. The use of the fathometer to show 
feeding habits and examination of the physiological differences 
of the species to show particular adaptation to different habits 
will be a tremendous step in enriching the students knowledge 

of marine biology. 

4. It is the intent of the pilot program to not limit its services 
to only college preparatory students. Students of all ability 
levels from the participating districts and schools will be en- 
couraged to take advantage of the program. 

5. It will be the responsibility of the participating district or 
school to provide transportation of its students to and from the 
dock from which the marine laboratory boat departs. 

6. The Orange County Superintendent of Schools Office will act as 
scheduling coordinator and contact point for the districts and 
schools. The participating districts will make application to 
this office through the coordinator for science and secondary 
education. As many schools and classes as possible will be 
accomodated as time and facilities permit. 

Objectives of the Floating Science Laboratory. 

1. Utilize all possible talents and resources to develop an 
unusually valuable and meaningful educational experience 
for a broad segment of junior and senior high school youth. 

2. Develop student appreciation and understanding of the sea, 
and life in the sea as a vast, largely untapped, natural 
resource. 
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3. Extend student awareness of the need to apply sound conser- 
vation practices in the use of the ocean and its many resources. 

4. Provide students with a "hands-on" experience in the handling 
and use of scientific equipment related to the several branches 
of marine science. 

5. Improve students’ understanding for the need to bring the 
tools and concepts of many scientific disciplines to bear on 
the problem of investigating the secrets of the sea. 

6. Assist students in the investigation of the educational and 
vocational opportunities associated with marine science. 

7. Encourage the development of marine science as a permanent 
part of educational programs of participating schools and 
school districts. 

8. Produce materials of instruction of district classroom use, 
K-12, for orientation, biological and physical science programs. 

Evaluation 

The ability to make the on board experience available to 
more than the original number of students ranks number one in 
results exceeding expectations. The original grant perspectus 
outlines the activities for some 5250 students from public and 
parochial schools in this county. However, because of many 
factors, the most notable being the acquisition of a superior 
vessel by means of going to public bid in the summer prior to 
commencing the program, the program was able to offer the 
marine science experience to over 8,000 students and 300 
teachers . 



Another result that exceeded the program expectation was 
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in the publication of 11,000 copies of the Marine Science Student 
Syllabus. Through the cooperation of the County Schools Office, 
Division of Publications and the Audio Graphics Department, these 
volumes were ready and made available prior to October. 

Another outstanding result of the program has been the pro- 
duction of a 21 minute, 16 mm color, sound film depicting the on 
board program. The total cost of the film was $2000. 

Through the development of an extension program, making 
cruises available on Saturdays on a subscription basis to schools 
not participating in the normal program, an additional 800 students 
and teachers were able to share in the on board experiences. 

The development of the Extension Program has enabled the 
Floating Laboratory to offer its services beyond the County. 

The extension program has allowed, therefore, the opportunity 
to lay the ground work for preliminary communications regarding 
joint power agreements for continuation of the program after 
federal assistance terminates. 

Another result of the program that has exceeded expectations 
has been the development of an Onshore Laboratory. Through the 
cooperation of Davey’s Locker, Inc. and the Ducommun Company a 
room was donated to the program for use as an Onshore Laboratory 
facility. 

Davey’s Locker negotiated with the Ducommun Company, owners 
of the Balboa Pavilion, and a room in the Balboa Pavilion was 
offered to the Floating Laboratory program. The program director 
feeling that the on board program was running smoothly made avail- 
able the onshore facility to the county’s schools. The schedule 
was filled two weeks after the general announcement was made. 

The onshore laboratory is therefore operated for the purpose of 
providing students who may not participate on the Floating 
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Laboratory an experience in the marine sciences. This operation 
is conducted at absolutely no cost to the program or the par- 
ticipating schools other than they are required to provide their 
own transportation. Over 1500 students will share this exper- 
ience by the end of the year. The Floating Laboratory provides 
specimens for the onshore laboratory and the participating schools. 

The one major problem of the program has been its inability 
to provide the experiences to more students. 

Another area of concern that had not measured up to expec- 
tation is the teacher orientation program. The program with 
its small staff did not have the ability to do an adequate 
in-service or follow-up program development during the first 
semester . 

This situation was slightly reduced during the second 
semester as a result of the program’s development and the degree 
of impact from the exposure of over 150 teachers during the first 
semester. Their in-school assistance to teachers participating 
in the program has been tremendous. 
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4. Letters To Paice Directors 
a. Introduction 

Letter were drafted and sent to a carefully selected set 
of PACE Project Directors of science programs to obtain per- 
tinent data to be included in the relevance guide and the rel- 
evance network. This data was requested from project directors 
throughout the United States to provide a comprehensive data 
collection. 

The third level of the relevance network was the program 
level which was composed of the Mobile Science Laboratory 
Program and nine other programs selected out of all the pro- 
jects received from these letters. Each program brochure re- 
ceived was analyzed to determine which ones most resembled the 
functions of the Mobile Science Laboratory Program, Ten were 
selected as options and included in the program level of the 
relevance network. Also, a comprehensive description of each 
program was included in the relevance guide to assist the 
voters in relevance assignment. 

The MSL project director selected the project directors 
out of the ’Pacesetters in Innovation’ summary, ALJ Associates, 
Inc. typed and forwarded the following letters to each director. 
The list of directors queried is also included. 
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September 27, 1968 



SC'HOOI DISTRK TNo. 241 
Albert Lc j, Minnesota 
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Mr. Orien C. Shockley 

Superintendent, Santa Fe Public Schools 
Santa Fe, New Mexico 87501 

Dear Mr. Shockley: 

I have noted with great interest the description of your Outdoor 
Education Program recently described in the SEIAC Eric Newsletter of 
April 1968. I feel that information on your program. Cooperative 
Project to Provide Supplemental Services to a Group of Elementary 
and Secondary Schools of New Mexico, would help us on our Mobile 
Science Laboratory facility. 

I would be interested in obtaining information concerning your 
program that you may have available, ie, copies of your proposal to 
OE and any descriptive materials. In addition, we would like your 
candid opinion of the effectiveness of the various aspects of your 
program. Do you feel that the original objectives of the first 
proposal are being attained? Have you implemented any changes in 
your program since the original proposal that have improved it. 

We are presently in our third year of operation with the Mobile 
Science Laboratory facility. This Title III program is considering 
an evaluation program as a part of the last year’s operation. 

We have prepared two brochures on our own operational program 
and would be happy to provide copies of them to you if you so desire. 

Sincerely yours. 



Sample Letter 
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Brooksidc Jr. High School 



DIRECTOR 




September 27 , 1968 A lbcr( 1 1 .t, M inncsota 5600 



Mr. James M. Riley, Coordinator 
Multi-Media Instructional Center 
1115 West Hillsboro 
B1 Dorado, Arkansas 71730 

Dear Mr. Riley: 

I have noted with great interest the description of your Title 
III education Program recently described in a PACESETTERS IN INNOVATION 
summary. I feel that information on your program, Multi-Media 
Instructional Center, would help us on our Mobile Science Laboratory 
facility . 

I would be interested in obtaining information concerning 
your program that you may have available, ie, copies of your proposal 
to OE and any descriptive materials. In addition, we would like 
your candid opinion of the effectiveness of the various aspects 
of your program. Do you feel that the original objectives of the 
first proposal are being attained? Have you implemented any changes 
in your program since the original proposal that have improved 
it? 



We are presently in our third year of operation with the Mobile 
Science Laboratory facility. This Title III program is considering 
an evaluation program as a part of the last year’s operation. 



We have prepared two brochures on our own operational program 
and would be happy to provide copies of them to you if you so desire. 

Sincerely yours , 
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Sample Letter 
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b. Mailing List of Project Directors 



Mr. Harold L. Coles, County Superintendent of Schools 
Central California Laboratory for Learning-Extension 
2314 Mariposa Street 
Fresno, California 93721 

Mr. J. Win Payne, Superindendent 

Experimental Forest 

Napa Valley Unified School District 

1750 First Street 

Napa, California 94558 

Mr. Blaine Wishart, Executive Director 
Superior California Educational Resources Agency 
1854 Fulton Avenue 
Sacramento, California 95825 

Mr. Cliarles R. Baker, Director 

Conservation, Recreation and Outdoor Science School (Project CROSS) 
Pupil Personnel Service and School Psychologist 
P. 0. Box 518 

San Andreas, California 95249 

Mr. Wesley D. Gordon, Coordinator of Science 
Natural History Museum and Research Center 
San Lorenzo Unified School District 
15510 Usher Street 
San Lorenzo, California 94580 

Mr. F. A. Grunefelder 

Orange County Superintendent of Schools Office 
1104 West Eighth Street 
Santa Ana, California 92700 

Mr. Gaylord A. Nelson 

Northern San Joaquin Valley Counties 

Supplementary Education Center 

222 Hast Weber Avenue 

Stockton, California 95202 

Mr. Roy G. Brubacher, Executive Director 
Cooperative Summer School Camp 
San Luis Valley Board of Cooperative Services 
Alamosa, Colorado 81101 

Mr. Roy G. Brubacher, Consultant 
Cooperative Summer School Camp 
Boards of Cooperative Services 
Adams State College 
Alamosa, Colorado 81101 

Mr. John G. Stuart, Superintendent 
Adams County School District 14 
4720 East 69th Avenue 
Commerce City, Colorado 80022 
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Mr. Donald P. LaSalle, Science Coordinator 

Talcott Ridge Science Center for Student Involvement 

Avon Board of education 

Avon Junior-Senior High School 

West Avon Road 

Avon, Connecticut 06001 

Mr. C. Fred Graef, Chief School Officer 
Pilot Nature Center Program 
Greenwood Consolidated School, No. 91 
Greenwood, Delaware 19950 

Mr. Wilmer E. Shue, Superintendent of Schools 
Outdoor Laboratory 
83 East Main Street 
Newark, Delaware 19711 

Mr. Buford H. Galloway 

Superintendent of Public Instruction 

Columbia County Board of Public Instruction 

P. 0. Box 1148 

Lake City, Florida 32055 

Mr. James E. Hall 

Superintendent of Public Instruction 
Escambia-Santa Rosa Humanities Curriculum Center 
P. 0. Box 1470 
Pensacola, Florida 32502 

Mr. Alan E. Hart 

Superintendent of Public Instruction 
Resource-Use Outdoor Education Center 
Taylor County Board of Public Instruction 
P. 0. Box 509 
Perry, Florida 32347 

Dr. Thomas W. Gulford 

Superintendent of Public Instruction 

Sarasota County Educational Enrichment Centers 

2418 Halton Street 

Sarasota, Florida 33577 

Dr. Lewis Shelton 
Fernbank Science Center 
DeKalb College 
555 N. Indian Creek Road 
Clarkson, Georgia 30021 

Mr. William Thomas, District Superintendent 
827 Fort Hall Avenue 
American Falls, Idaho 83211 
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C. Computer Application 

1. Ifitioduction 

This computer section defines the computer program input/output 
system used on the subject contract. The complete input/output 
technique is delineated in the I/O flowchart. An example set of 
data listings is shown for each phase of the program. 

2. loput/outpuc Flowchart Description 

The computerized evaluation methodology can be divided into 
three distinct phases: 

The first phase, as shown in Figure C-1, consists of programs 
'FT 01, 'FT 02', 'FT 205' and 'FT 03'. The input to the first 
program is option and criterion data. The program converts the 
input data to magnetic tape records and computes level numbers 
from the coded options. 

The records are sorted sequentially on the first twenty nine 
positions which are the criterion number, option code, card code 
and the level number respectively. The tape-records are then 
inputed to 'FT 02. 'FT 02* combines the code 1 and 2 records 
and merg'es the criterion record data with the appropriate option 
record dat=i creating a new tape file. The 'FT 02* tape records 
are sorted on the first twenty-seven positions which are the 
level number, option ID, card code and the criterion number 
respectively. *FT 205* performs a file maintenance on the output 
records of 'FT 02' and the records are then inputed to 'FT 03*. 

'FT 03' produces the blank ballot matrices used to assign 
relevance numbers. 
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Figure Ol. Input/Output Flowchart, Phase I 
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The second phase, as shown in Figure C-2, consists of pro- 
grams ’FT 04' and 'FT 05. 'FT 04* performs a validity check of 
the assigned ballot relevance data and converts the data into 
the proper format for the output tape records. The output 
tape-records are then sorted on the first fifty-six positions 
which include the level number, base ID, voters initials, voter 
organization, and the option ID. The tape-records then become 
the input to 'FT 05'. 'FT 05' normalizes the assigned option 
and criterion values, computes Individual and Average Relevance 
and the Percent Standard Deviation. Individual and Average 
Relevance and the Percent Standard Deviation are outputed in 
printed listings. The program also, produces a tape-file that 
is sorted on the first twenty- four positions which are the level 
number and the option ID. 

The third phase, as shown in Figure C-3, consists of the pro- 
grams 'FT 06, 'FT 07' and 'FT C8. 'FT 06' computes the Branch 

Relevance and creates the BR file. The BR file is the file used 
to maintain the Branch Relevance records. The BR records are 
sorted on the first twenty-nine positions which are the level 
number. Branch Relevance field and the ID code and inputed to 
'FT 07'. 'FT 07' ranks the elements in two formats first by 
relevance and then by ID, and creates an output tape- file. The 
output tape-file records are sorted on the first twenty-nine 
positions which are the format code, level number and the 
relevance- ID field. The records are then inputed to 'FT 08' which 
produces the Branch Relevance listings. 

3. Representative Computer Outputs 



The first output of the computer programming system is Blank 
Ballot Matrices used by the students and educators in assigning 
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option and criterion values. Figure C-4 is an example of a ballot 
printed by the computer for the Mobile Science Laboratory Eval- 
uation. The title at the top of the ballot reflects the node of 
interest. The ID's and titles under the node title are represent- 
ative of the options to the node. Along the top of the matrix are 
the criterion ID's, under each a criterion value is placed when 
balloting the node. Below the criterion weight row on the left 
column are the ID's reflecting the options which are also, 
assigned values for each criterion. Values are assigned by team 
members in sets. A set consists of the complete criterion row 
(left to right) or a column (up and down) of values below each 
assigned criterion weight. The values can be inserted in three 
formats as long as they are uniform on each individual ballot. 

The value can be in the format of (x.x) , (xx.), or ( .xx) . The 
program will handle all three formats and normalize to unity 
in 'FT 05'. Below the matrix are the criteria ID's with their 
appropriate titles to aid the voter in identifying the criteria 
when assigning values to the options. 

The second type of output listing of the PATTERN system is 
the Individual Relevance, Average Relevance and Percent Standard 
Deviation. Figure C-5 is an example of an Individual Relevance 

listing. The listing is composed of a heading. Individual 
Relevance Matrix, Name and Organization of the voter. Under the 
headings are the criterion weights normalized, by row, to unity 
by the computer. Below the criterion values, on the left margin, 
are the option ID's, the center columns are the individual assigned 
relevances normalized, by column, to unity. The right-most column 
of values is the Individual Node Relevances. 
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Figure C-6 is an example of an Average Relevance and Percent 
standard Deviation listing. The listing is composed of the 
heading, Average Relevance Matrix below which are the option ID's, 

the Average Relevances and the Percents Standard Deviation respec- 

% 

lively . 

The Individual and Average Relevance output listings are used 
by the evaluation team to determine agreement of individual bal- 
loting at a node. The Percent Standard Deviation reveals the 
actual degree of agreement and aids in determining problem areas 
in either option or criterion linkage to the overall relevance 
network. 

The third output listing of the PATTERN system is the Ranked 
Branch Relevance listings, produced by the Branch Relevance Phase. 
The listings are printed by the computer in two formats. The 
first format as shown in Figure C-7 is a listing of option ID's 
and Titles ranked according to the Branch Relevance number. The 
second format, Figure c-8 is a listing of option titles and Branch 
Relevance numbers according to ID. The listings are utilized by 
the evaluation team for analysis and data files. 
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Figure C-6. Sample Average Relevance and Percent Standard Deviation Listing 
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D. Student Project Cards 

1. Introduction 

Four types of data gathering cards were designed to assist 
the students in organizing in detail the entirety of their pro- 
jects as well as to assure maximum compatibility of data collected 
with the Mobile Science program’s evaluation methodology. The 
four types of cards, (1) Project Summary Card, (2) Project Descrip- 
tion Card, (3) Supporting Data Card and (4) Criteria card were 
designed prior to the 1968 summer program for student use in 
collecting data. Cards were used instead of paper to facilitate 
data collection in the field. 

This section explains the rationale for use of each type of 
card, the disposition of data on the cards, the problems arising 
in use of the cards, and the recommendations for future card usage. 

2. Pfojecc Sumintry Card 

The Project Summary Card as shown in Figure D-l is an abstract 
of the students project. There is one Project Summary Card for 
each project. The front of the card is pre-printed and is used to 
define the broad concepts of the project in outline format. The 
project number is placed in the upper left corner and the title in 
the center under the project number. The pre-printed information 
are the column headings i planning, implementation and analysis. 

Planning is subdivided into five sections: what, how, when, 

where, and why. The planning part of the outline is designed to 
help the student organize completely all his thoughts and ideas 
about his anticipated modus operandi. 

Implementation is subdivided into predictables and unpredict- 
ables as a minimum, plus any other pertinent data selected by the 
student. The implementation part is designed to record expected 
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Figure D-1. Project Sumiiiary Card 
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and unexpected happenings while on the field trip. 

Analysis also consists of five subdivisions: model, data 

results, conclusions, and recommendations. The model is a 
definition or identification of the student selected variables. 

The data are the analysis of all relevant material collected, 
irrelevant data are deleted. The results include appropriate 
graphs, charts and comparisons with description and application. 
The conclusions specify whether the project supported planned 
objectives and pro j ections , dissents, or is inconclusive in 
nature. The recommendations reveal new ideologies for future 
projects as well as criticisms of past program experiences. 

The back of the card is used for a short description or 
summary of the planned aspects of the project. The back of 
the card is also used to define or continue the elements of 
the outline on the front side. 

3. Project Description Card 

The Project Description Card as shown in Figure D-2 is 
similar in format to the Project Summary Card. This card is used 
to further sub-divide the outline elements of the Project Summary 
Card and also to further sub-divide other Project Description 
Cards. The main purpose of this card is to try to get the 
student to break down every element in his or her outline to 
their smallest component parts and be aware of their interrelated- 
ness. The cards are also of benefit in developing in the students 
a systematic approach to research and study. 

There can be as many Project Description Cards as a student 
deems necessary or appropriate to sub-divide his or her project 
into its component parts. The planning, implementation, and 
analysis are the only three given categories of project outline. 
The students own motivation dictates the extent of outline 
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element breakdown. • 

i 

4. Supp<^.ft Data Card 

The Supporting Data Card as shown in Figure D-3 is used to 
record data that supports, defines, or compliments outline elements 
on the Project Description Cards. All data collected that does not 
lend itself to outline format such as charts, graphs, diagrams, and 
application explanation are recorded on these cards . There can be 
as many Supporting Data Cards used as a student wishes to record 
the data pertinent to his or her project. 

5. Criteria Card 

The Criteria Card as shown in Figure d- 4 is used to record the 
students reasons for doing each part of his or her project. This 
card works directly with the Project Description Card in that while 
the student is developing a systematic approach to research, he 
is also recording the reasons for each and every step. 

The criteria cards are benefical to the students in that many 
times after careful consideration, they find easier methods of 
program accomplishment. In other instances, they feel they wasted 
time in the field or that they were not properly prepared. 

The educators benefit also from the criteria cards in that 
they are better able to evaluate the student’s project when they 
know exactly how and why the student performed. 

6. Coding of Cards 

Considerable effort and time were spent coding (assigning 
representative alpha-numeric characters) each element of the pro- 
ject outline. The elements on the project outline are coded to 
assure compatabi li ty with the computer program aspect of the 
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evaluation methodology. 

The computer program identifies each element of the outline 
network by a particular code. Planning is an A code, Implemen- 
tation is a B code, and Analysis is a C code. Under planning 
A1 equals what, ^ equals how, ^ equals when, M equals where, 
and ^ equals why. The next element outline breakdown under M 
would be All which would subdivide into Alll and so on for each 
outline element of the what under planning. The same method of 
coding is used for implementation and analysis. Each subsequent 
element in a category uses as the base for its code, the code of 
.the preceeding element. 

Figures D-5 and D-6 are a coded Project Summary Card and Pro- 
ject Description Card from project 33, "A Comparison of Two Northern 
Minnesota Forest Communities", as coded 1A01F2304433 . The code 
means: Extended Equipment or Technique (1), In the Science area (lA) 

Using the Mobile Science Laboratory (lAOa) , At the Secondary 
Education level (1A01F2), In the Life Science Curriculum (1A01F23), 
Project assigned to the 4th group (1A01F230£) , Student having 
participated in the MSL program four years and the project number 
33 (1A01F230433) . Each subsequent breakdown uses as the base for 
its code, the code of the preceeding option. The code (1A01F230432) 
is a base for each outline element breakdown of project 33. For 
example, 1A01F230433B2 represents unpredictables in project 33, 
and 1A01F230433C4 represents conclusions, and 1A01F230433C41 
represents the next subsequent element in conclusions, etc. 

f 

7. Method of Conversion to Tape 

The data collected on cards in the field by the students 
were typed and formated for magnetic tape storage. The reason 
the data were put on magnetic tape was for IBM MT/ST utilization. 

The MT/ST records all the typed data on magnetic tape and will 
play it back automatically. All corrections, deletions, or 
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additions can b© mad© without having to r©type th© complete pro- 
ject. The only typing required after th© original typing is 
adding or correcting, deletions are done on the console. The 
tapes are also very easy to store, transport and maintain. Any 
time a particular project is required, the tape storing the 
project can be placed in the MT/ST and the project data will be 
automatically typed out. 

In the Mobile Science Laboratory evaluation, the MT/ST was 
invaluable. Every project had to be typed, then the typed copies 
were sent to the students for editing and correcting when 
necessary. The edited copies were then returned for corrections. 
Correcting the projects consisted of playing out the tape con- 
taining the project; deleting as applicable, adding or correcting 
grammar and content. The portions of the typed data that were 
unchanged automatically typed out. 

The 50 magnetic tapes after being used for correcting student 
projects are a permanent, easily accessible data bank that was 
provided to the MSL Program as part of the contract. They can 
be used by the Mobile Science Laboratory committee for demon- 
strating purposes, examples of student projects, and compact 
storage containers. They can be modified at any time and still 
maintain their desired content without re-typing the entirety 
of the data. 

8. Problems in Student Card Usage 

Many project cards were hard to read due to lackadaisical 
writing and use of faint lead pencils. Many parts of outline 
elements were so terse it was difficult to comprehend the exact 
meaning. The students, in many instances were not sure where 
the data collected were supposed to be placed on the cards . 

Many students felt that they spent too much time on their cards 
and not enough time on data gathering. Many students felt that 
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the cards should have been larger to allow more room for element 
qualification. A large number of the students were not aware of 
exactly how to use their cards properly. 

These difficulties are to be expected however, since this was 
the first year for the students using cards. Many of the students 
participated this year in the MSL program for their second, third, 

or fourth time and the change in procedure was unfamiliar. 

9. Recomfflendadons for Card Usage 

The students should use ink whenever possible and print all 
the information. They should be more specific on each element 
of the outline network (three word minimun) i.e. comparison of 
soil technique, versus just 'soil' for an outline element. 

The use of each type of project card should be explicitly 
defined and illustrated to enhance student comprehension. The 
cards should possibly be larger to allow the students more space 
for element qualification. 

Some students would like the cards standardized with a 
definite place to enter all data gathered in the field. However, 
many students liked the cards exactly the way they were. Possibly, 
there could be more pre-printed information on the cards so the 
students would know exactly where to put their data. 

One general comment in the card usage is to have the data 
typed immediately after the field trip so the students can edit 
the material while the facts are still clear in their mind. 
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E. Sample Project Write-up 



1. Format Description 

A Comyarison of Two Northern Minnesota Forest Communiti^es 
number ( 1A01F2304433) is an example of a completed project as 
typed from the MT/ST magnetic tape. 

The first page of the project is a direct conversion of 
Figure D-1 (Project Summary Card). The project summary, planning, 
implementation, analysis, and each of their sub-elements up to 
the colons are the front part of the card. For example, in the 
planning category, A1 Comparison: and analysis, C4 Conclusions:. 

The card itself limited further information at this level of 
element break down. From the colon to the end of the phrase 
appears on the back of the summary card as shown in Figure D-1. 

For example, the front side, A2 Methods :back of card, mode or 
procedure. This method was designed to counteract card space 

limitations . 

At the bottom of the first page and the top of the second 
page of the project is the conversion of the Project Description 
Card, Figure D-2. The students subdivided A1 Comparison into its 
component parts of All Gunflint Trail and A12 Itasca and further 
divided Gunflint and Itasca into their component parts. As on 
the Summary Card, the space limitations required the definitions 
or qualifying remarks of each subdivision of Gunflint and Itasca 
to be placed on the back of Project Description Card as shown 
in Figure D-2. 

The Supporting Data and Criteria are Section II and III of 
this project. Section IIA is a direct conversion of the Sup- 
porting Data Card, Figure D-3. The charts are too involved and 
required too much space to be written on the Project Description 
Cards. Therefore, they are placed on Supporting Data Cards and 
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referenced through the qualifying remarks at the top of the 
card. For example, "First Transect, July 30, 3:00 P.M., 

Transect Three". 

The criteria, Section III of the project are a direct con- 
version of Figure D-4. The criteria are qualified by their 
preceding code. For example, "A Planning of a project .. .etc." 
references the planning on the Project Summary Card. These 
criteria or explanations are why a student performed a particular 
operation in carrying out his or her project. In this instance 
the project is a group project. 










2. Project 

A COMPARISON OF TWO NORTHERN MINNESOTA FOREST COMMUNITIES 

Project No. 1A01F2304433 

I PROJliCT DESCRIPTION 

Project Summary:The purpose of this project is to compare (ecologically) 
two northern Minnesota forest communities. One community is in 
Itasca State Park on an island in Sc^uaw Lake. The other community 
is 600 ft. north of camp #712 on the Gunflint trail, Grand Marais, Minn. 

In both communities, similar methods were employed to study veg- 
etation, soils, insects, and small mammals in a 600 ft. by 400 ft. area. 

The data will then be analyzed by means of comparisons between 
the two communities. 

A. Planning: scheme of procedure for carrying out the study. 

A.l. Comparison: to illustrate the similarities and differences 
between Gunflint and Itasca. 

A. 2 . -Methods : mode of procedure. 

A. 3. July 12 - Aug 3, 1967 July 15-Aug 2, 1968 
A. 4. Location : areas designated for the study. 

A. 5. Objectives : reasons for doing the study. 

B. Implementation : execution of the planning. 

B. l. Predictables : planned or foreseen events or factors. 

B. 2. Unpredictables : unexpected events or factors. 

C. Analys is : determing essential features of the study. 

C. l. Model : pattern. 

C.2. Data:,informatioii collected during the study. 

C.3. Results : outcome of the study. 

C.4. Conclusions : summing up of the results. 

C.5. Recommendations: other expedier. ts. 

A.l. Comparison: to illustrate the similarities and differences of the 
two areas. 

A. 1.1. Gunflint Trail 



A. 1.1.1. Insects: any small invertebrate having a segmented body 
and six legs. 

A. 1.1. 2. Vegetation : sum total of plant life. 

A. 1.1. 3. Soilisurface material of the earth. 

A. 1.1.4. Small mammals : members of the class Mammalio, Phylum 
Chordata, order Rodentia 
A. 1.2. Itasca 

A. 1.2.1. Insects 
A. 1.2. 2. Vegetation 
A. 1.2.3. Soil 
A. 1.2. 4. Mammals 

A. 2. Methods :modes of procedure, the same for 1967 and 68 studies. 

A. 2.1. Grid: area within which the sampling to be done. 

A. 2. 1.1. 600' X 400' area. 

A. 2. 1.2. 100' X 100' quadrat:area for sampling, 

A. 2. 1.3. Forest:area of study. 

A. 2. 2. Vegetation 



A. 2 . 2 . 1 . 
A. 2 . 2 . 2 . 



A. 2. 2.3. 



A. 2. 2.4. 
A. 2 . 2 . 5 . 



3 40' transects/quadrat 

Collect : unknown types for identification and reference. 
Press :place specimens under pressure to dry for pre- 
servation . 

Identify: use keys to find the names. 

Record: types and numbers found. 



A. 2. 3. Soil 



A. 2. 3.1. Collect : sample soil at 20' and 40' along the plant 
transect . 

A. 2. 3. 2. Information : humus depth and type. 

A. 2. 5. 3. Analysis: pH, phorphorus , nitrogen, potash content. 
A. 2.4. Insects 



A. 2. 4.1. 5 sweeps, with to a transect 
A. 2.4. 2. Put insects caught in killing jars 

A. 2. 4. 3. Mount : preserve specimens by pinning them to a specia 
board . 

A. 2. 4. 4. Identify: use keys to find the names. 

A. 2. 5. Mammals 
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A. 3. 



A. 2. 5.1. 1967 
A. 2. 5. 2. 1968 
A. 2. 5.1. 1967 

A. 2 . 5 . 1 . 1 . 
A.2.5.1.2. 
A . 2 . 5 . 1 . 3 . 
A. 2. 5. 1.4. 
A. 2. 5. 2. 1968 
A. 2. 5. 2.1. 
A.2.5.2.2. 
A. 2. 5. 2. 3. 
A. 2 . 5 . 2 .4 . 
A . 2 . 5 . 2 . 5 . 



methods : uti 1 i zed in the 1967 study 
methods : utilized in the 1968 study 
methods 

600’ X 400’ grid. 

Stations every 25’ with traps. 

3 traps at each station. 

Collection twice daily 
methods 

300’ X 500’ grid. 

Station every 50’ with traps. 

2 traps at 1 station. 

Collection twice daily 
Identificationruse reference books 



at Itasca, 
at Gun flint. 



to find the 



names . 



A. 2. 3.1. Collect : collect soil samples to find pH, nitrogen, 
potash phosphorus content. 

A. 2. 3. 1.1. Quadratrarea designated for study. lOO’xlOO’ 
study area witliin the 600’x400’ grid. 

A. 2.3.1. 2. 20’ and 40’ on plant transect 

A. 2. 3. 1.3. Dig to soi 1 : necessary to dig through the litte 

A. 2. 3. 1.4. Dryrsamples left exposed to the air overnight. 

A. 2. 3. 2. Information: data which was collected. 

A. 2.3. 2.1. Humus depthrhow far the soil was from the 
surface . 

A. 2. 3. 2. 2. Type of litter 

A. 2. 3. 3. Analys is : determination of essential features. 

A. 2.3.3. 1. Soil analysisrfor pH, nitrogen, phosphorus 
and potash content. 

Schedule : timetable for the project. 



A. 3.1. 1967 

A. 3 . 1 . 1 . 4 weeks 
A. 3. 2. 1968 

A. 3 . 2 . 1 . 4 weeks 

A. 4. Location : areas designated for the study. 
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A. 4.1. Itasca State park 
A. 4. 1.1. Papoose Island 
A.4.1.2. Squaw lake 
A. 4. 1.3. Forestrscrub 



A. 4. 2. Gunflint trail 

A. 4. 2.1. Forest : northern conifer 
A.4.2.2. 600 ft. north of camp #712 
A. 4. 3. Brooks ide 



A. 5. Obj ectives : reasons for doing the study. 

A. 5.1. #1 

A. 5. 1.1. Gain knowledge 

A. 5. 2. # 2 

A. 5. 2.1. Detect similarities and differences between the Itasca 
and Gunflint study areas. 

A. 5. 3. #3 

A. 5. 3.1. Needed project 

B. l. Predictables 

B. 1.1. Schedule 

B. l. 1.1. 1967 

B.l. 1.2. 1968 schedule: the first week there we had lectures 
by various people working in the Gunflint area to 
acquaint us with it. The afternoons of the second 
and third weeks were given to setting up and then 
studying the gird. The cards were worked upon in the 
evening . 



B.l. 2. Data 

B.l. 2.1. 1967 
B.l. 2.2. 1968 
B.l. 3. Data format 
B . 1 . 3 . 1 . Soils 
B . 1 . 3 . 2 . Insects 
B.l. 3.3. Plants 
B.l. 3.4. Mammals 




B.l. 2.1. 1967 data 
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B. 1.2. 1.1. 


Plants 


B. 1.2.1. 2. 


Soil 


B. 1.2. 1.3. 


Insects 


B. 1.2. 1.4. 


Mammals 


.2.2. 1968 


data 


B. 1 . 2. 2 . 1. 


Plants 


B. 1.2. 2. 2. 


Soil 


B. 1 . 2. 2 . 3. 


Insects 


B. 1.2. 2. 4. 


Mammals 



B.1.3. Data format : sheet which shows the general type of infor- 
mation to be collected. 

B. 1.3.1. Soil 

B.1.3. 1.1. Humus depth 
B.1.3. 1.2. ph 

B.1.3. 1.3. Nitrogen content 
B.1.3. 1.4. Potash content 
B.1.3. 1.5. Phosphorus 
B.1.3. 2. Vegetation 

B.1.3. 2.1. Date of collection 
B.1.3. 2. 2. Time of collection 
B.1.3. 

B.1.3. 2. 3. Location collected 
B.1.3. 2. 4. Plant type 
B.1.3. 2. 5. Number of plants collected 
B.1.3. 3. Insects 
B . 1 . 3 . 3 . 1 . Date 
B.1.3. 3. 2. Time 
B.1.3. 3. 3. Location 

B.1.3. 3. 4. Weather at time of collection 
B . 1 . 3. 3 . 5. Lab work 
B.1.3. 3. 6. Type of insect 
B.2. Unpredictab les 
B.2.1. New factors 
B.2.1.1. Weather 
B.2.1. 2. Accident 
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B . 2 . 1 . 3 . New tool 
B.2.1.4. Error 
B.2.2. Different values 
B.2.2.1. Off schedule 
B.2.2. 2. Conflicting data 
B.2.1. New factors 
B.2.1.1. Weather 

B.2.1. 1.1. Temperature 
B . 2 . 1 . 1 . 2 . Rain 
B . 2 . 1 . 1 . 3 . Wind 
B.2.1. 1.4. Clouds 
B.2.1. 2. Accident 

B.2.1. 2.1. Broken instrument 
B.2.1. 2. 2. Lost data sheet 
B.2.1. 2.3. Sickness 
B.2.1. 2. 4. Lost pencil 
B.2.1.3. New tool 

B.2.1. 3.1. New reference 
B.2.1. 3. 2. More effective method 
B.2.1.4. Errors 

B.2.1. 4.1. Bad reading 

B.2.1. 4. 2. Improper documentation 

B. 2.2. Different values 

B.2.2.1. Off schedule 

B.2.2. 1.1. Load too heavy 
B.2.2. 1.2. Load too light 
B.2.2. 1.3. Interruptions 
B.2.2. 1.4. Resource not available 

B. 2.2. 2. Conflicting data 

B.2.2. 2.1. Illogical values 
B.2.2. 2.2. Disagreement with theory 
C.l. Model 

C. 1.1. Plants 

C. l. 1.1. Coefficient of community 
C.l. 1.2. Diversity index ratio 












I 
I 

D 
D 
D 
D 
0 
0 
Q 
0 
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C.1.1.3. Dominance 
C.1.2. Insects 

C. 1.2.1. Coefficient of community 
C.1.2. 2. Diversity index ratio 
C.1.2. 3. Dominance 
C.1.3. Soil 
C.1.4. Mammals 
C.2. Data 

C.2.1. Selecting 

C.2.1.1. Relevant data 

C.2.2. Sorting 

C.2.2.1. Comparison 
C.2.3. Presentation 
C.2.3.1. Written 
C.2.3. 2. Numerical 
C.2.3. 3. Research paper 

C.3. Results 

C.3.1. Plants 

C.3.1.1. Model 

C.3.1. 2. Number of individuals 
C.3.1. 3. Number of species 
C.3.2. Insects 

C.3.2.1. Number of individuals 
C.3.2. 2. Number of species 
C.3.3. Soil 
C.3.3.1. pH 

C.3.3. 2. Nitrogen content 
C.3.3. 3. Potash content 
C.3.3. 4. Phosphorus content 
C.3.4. Mammals 
C.1.1. Model plants 

C. 1.1.1. Coefficient of community 

C.1.1. 1.1. Expression of similarities of 

C.1.1. 2. Diversity index 



ies lists 



m 
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C. 1.1. 2.1. Ratio between number of species and number of 
individuals 
C.1.1.3. Dominance 

C. 1.1. 3.1. Most numerous plants 
C.4. Conclusions 
C.4.1. Plants 
C.4.2. Insects 
C.4.3. Soil 
C.4.4. Mammals 
C.5. Recommendations 
C . 5 . 1 . None 

II SUPPORTING DATA for 1968- -Gunf lint Trail 



A. First transect July 30 3:00 PM transect 3 



plant type 
wild sarsaparilla 
Labrador tea 
Blindia 

Canada mayflower 
red osier dogwood 
jack pine 

Lycopoduim clavitum 

bunchberry 

twinf lower 



Number counted 
3 
5 

2’ solid growth of moss 
34 
3 

1 

'> 

38 

28 



B. Second 
plant type 
bunchberry 
twinf lower 
Blindia 
Lycopodium 
violet 
large leaf 



transect July 30 3:00 PM 

Number counted 
44 
42 

11” solid growth 
complanatum 2 

1 

aster 12 



of moss 



' o 
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speckled adler 
Dicranum 
balsam 
blueberry 



3" solid growth of moss 
1 
2 



0 

D 

0 



0 



C. Third transect 
bunchberry 
twinf lower 
large leaf aster 
strawberry 
balsam 

reindeer moss 
black spruce 
Canada mayflower 
Blindia 

A. First transect 

Plant type 

Lycopodium clabatum 

wild sarsaparilla 

bunchberry 

reindeer moss 

Blindia 

twinf lower 

balsam 

spruce 

speckled alder 
Canada Mayflower 
black spruce 
red osier dogwood 
grass 



40 

17 

7 

1 

2 

3' 

2 

14 

20 ' 



solid growth of moss 



solid growth of moss 



2:45 PM July 29 Transect 4 
Number counted 
1 
4 
24 

scattered 

3' solid growth 

7 

3 
1 

4 

8 
1 
1 

52 



B. Second transect 
plant type 
Canada mayflower 



Number counted 
3 



o 

ERIC 
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speckled alder 

Dicranum 

spruce 

bunchberry 

Blindia 

Lycopodium complanatum 
twinf lower 
s travsfberry 
ground pine 
blueberry 
reindeer moss 



3 

1’ 10" solid growth 



1 

24 

2’ 6" solid growth 



1 






4 

5 
1 

s cattered 



C. Third transect 

bunchberry 

b a 1 s am 

s tr awberry 

ground pine 

speckled alder 

gras s 

may flower 

large leaf aster 

twinf lower 

red osier dogwood 

cedar 

A. First transect July 

bunchberry 

black spruce 

Canada mayflower 

grass 

reindeer moss 

Blindia 

strawberry 

wild sarsaparilla 



61 

1 

4 

7 
1 

40 

1 

8 
6 
1 
1 

30 1:30 PM transect 7 

25 
2 
7 

12 

1» 8" solid growth 
6" solid growth 
3 
1 



Lycopodium clavitum 
red osier dogwood 
one sided pyrola 



3 

2 

1 



B. Second transect 
plant type 
twinflower 
balsam 

violet 

speckled alder 
black spruce 
large leaf aster 
reindeer moss 
bunchberry 
wild sarsaparilla 
birch 

strawberry 

blueberry 

C. Third transect 
speckled alder 
willow 

low cudweed 
grass 

Lycopoduim clavatum 
golden avens 
wild sarsaparilla 
large leaf aster 
oak fern 
violet 



number counted 
1 
1 
1 
3 
1 
8 

2" solid growth 
2 
7 

3 

4 
1 



4 

1 

12 

4 
1 

5 
8 

17 

2 

1 



A. First transect July 29 1:30 PM 
Lycopodium complanatum 1 
bunchberry 21 

strawberry 2 



transect 9 
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red osier dogVNrood 
large leaf aster 
spruce 

black willow 
golden avens 
Canada mayflower 
B lindia 

B. Second transect 
plant type 
oak fern 
Blindia 
bunchberry 
paper birch 
spruce 
twinf lower 
birch 

ground pine 
large leaf aster 

C. Third transect 
Blindia 
strawberry 
large leaf aster 
bunchberry 
mayflower 
twinflower 
black spruce 
wild sarsaparilla 
b lueberry 
red raspberry 

A. First transect July 29 

bunchberry 

twinflower 



3 

6 

1 

2 

1 

2 

10' solid growth 

number counted 
1 

entire transect 
20 
1 
1 
1 
2 
4 
6 



entire 
1 
6 

26 
2 
4 
1 
1 
2 
1 

2:15 PM transect 10 
19 
1 
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Blindia 

Canada may flower 

northern honeysuckle 

black spruce 

large leaf aster 

balsam 

Dicranum 

birch 

golden avens 



2’ solid growth 
6 
1 
1 
27 
1 

1’ 2" solid growth 
2 
1 



B. Second transect 
plant type 

birch 

Blindia 

bunchberry 

twinf lower 

Canada mayflower 

balsam 

Dicranum 

red osier dogwood 
mountain holly 
strawberry 
ground pine 

C. Third transect 
bunchberry 
mayflower 
strawberry 
Blindia 

wild sarsaparilla 
violet 
blueberry 
large leaf aster 
paper birch 
mountain holly 



Number counted 
1 

entire transect 
40 
5 
9 
4 

3" 

1 

1 

9 

1 



41 

4 

1 

7’ growth 
3 
1 
2 
2 
1 
1 
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A. First transect 
spruce 
tw inf lower 
bunchberry 
Canada mayflower 
speckled alder 
Dicranum 
balsam 
strawberry 
Blindia 
Lycopodium complanatum 9 
wild sarsaparilla 2 

violet i 

carpet of bunchberry and twinf lower 



July 30 

5 
47 
53 

6 
5 

3" 

4 

13 

21’ solid growth 



2:20 PM transect 13 



B. second transect 

plant type 

bunchberry 

twinf lower 

spruce 

Blind! a 

Canada mayflower 
violet 

golden avens 
large leaf aster 
wild sprsaparilla 
reindeer and Blindia 
strawberry 
Lycopodium clavatum 
reindeer moss 



Number counted 
32 
19 
1 

4" solid growth 
25 
2 

4 
9 

5 

4’ solid growth 
1 
3 

5" solid growth 



C. third transect 

Canada mayflower 25 

twinf lower 10 
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bunchberry 


41 


violet 


1 


wild sarsaparilla 


1 


lesser Pyrola 


2 


strawberry 


4 


red osier dogwood 


3 


reindeer moss 


17" solid growth 


Blindia 


15" solid growth 


Lycopodium lavatum 


1 


A. First transect 


July 25 1:30 PM transect 15 


violet 


2 


twinf lower 


9 


bunchberry 


19 


strawberry 


3 


may flower 


8 


wild sarsaparilla 


3 


golden avens 


1 


Blindia 


20’ 5" solid growth 


Black spruce 


1 


B. second transect 


plant type 


Number counted 


Blindia 


entire transect 


twinflower 


14 


mayf lower 


9 


bunchberry 


14 


balsam 


1 


strawberry 


1 


red raspberry 


1 


C. Third transect 


Blindia 


entire transect 


bunchberry 


18 
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ground pine 2 
balsam 5 
may flower 4 
wild sarsaparilla 3 
strawberry 3 
Lycopodium complanatum 3 
twinflower 7 
birch 1 



A. First transect July 25 



black spruce 2 

bunchberry 16 

Canada mayflower 9 

balsam 4 

paper birch 1 

Lycopodium clavatum 1 

Blindia entire 

twinflower 9 



2:30 PM transect 17 



transect 



B. Second transect 
spruce 

balsam 
bunchberry 
twinflower 
wild sarsaparilla 
Canada mayflower 
quaking aspen 
ground pine 

C. third transect 
plant type 
Blindia 

spruce 
bunchberry 
s trawberry 



5 

2 

1 

14 

3 

5 

1 

1 

Number counted 
20' solid growth 
2 
8 
2 



black spruce 
may flower 
twinflower 
balsam 
Dicranum 
reindeer moss 



A. July 23 1:30 PM 


Transect 19 first trans 


Blindia 


entire transect 


bunchberry 


35 


Canada mayflower 


21 


twinflower 


2 


Lycopodium clavatum 


2 


blueberry 


3 


orchis 


1 


reindeer moss 


small patch 


wild sarsaparilla 


1 


strawberry 


1 


B. Second transect 


large leaf aster 


10 


Boletinus pictus 


1 


Blindia 


3' solid growth 


twinflower 


9 


bunchberry 


6 


Lycopodium clavatum 


5 


reindeer moss 


4" solid growth 


lesser pyrola 


1 


Blindia and reindeer 6" solid growth 


Canada mayflower 


3 


black spruce 


3 


strawberry 


2 


C. third transect 


plant type 


number counted 



1 

1 

3 

1 

8" solid growth 
4" solid growth 
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balsam 

Blindia 

grass 

Lycopodium clavatum 
twinflower 
strawberry 
bunchberry 
Polypelus frondosus 
reindeer moss 
Canada mayflower 
red raspberry 



2 

3* solid growth 
1 
5 
4 
7 

26 

1 

19’ solid growth 
2 



A. First transect 
Canada mayflower 
bunchberry 
reindeer moss 
Blindia 

common high bush 
b lueberry 
ground pine 
black spruce 
twinflower 
balsam 



July 23 2:30 PM transect 24 

14 
17 

entire transect 
entire transect 
1 
1 
2 
1 
4 
3 



B. Second transect 

Blindia 

twinflower 

mayflower 

bunchberry 

wild sarsaparilla 

cedar 

ground pine 
black spruce 



entire transect 
7 
7 

16 

1 

1 

1 
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C. Third Transect 

bunchberry 
twinflower 
may flower 
Blindia 
black spruce 



1 

12 

4 



9" solid growth 
4 



SUPPORTING DATA for 1967 at ITASCA 
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Pyrola secunda 
Salix nigra 
Gnaphalium uliginosum 
Geum strictum 
Gumnocarpium Dryopteris 
Petula papyrifera 
Rubus occidentalis 
Lonicera villosa 
Nemopanthus mucronata 
Pyrola minor 
Polypilus frondosus 
Betulea lutea 
Populus tremuloides 
Chrysopis mariana 
Boletinus pictus 



1 

2 

12 

12 

3 

10 

4 
1 
3 
3 
1 
2 
1 

119 

1 



Dominant plants are: 

1. Cornus canadensis 

2. Linnaea borealis 

3. Maianthemum canadense 

4. Chrysopis mariana 



PLANT STUDY RESULTS 



Diversity Index 
Gunf lint 

number of species 
number of indiv. 
Itasca 

number of species 
number of indiv. 



= U « 

1730 

* 22 * 

1097 









% 



.0205 



.0205 



V 



BBS 
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Coefficient of Community 
Number of species - -Gunf lint : 36 
Number of species- - Itasca: 22 



Number of common species: 6 

6 = 6 = 9.8% 

39 = 22 61 



Gunf lint : 
Number of 
Number of 

Itasca: 
Number of 
Number of 



species : 36 
indiv . : 1730 



species : 22 
indiv . : 1097 



Plant Conclusion 
Conclusion 1 

The similarities in the diversity index indicate that species 
numbers relations were about the same in each community. 

Conclusion II 

Diversity indix is low indicating little similarity in species 
types between Itasca and Gunf lint. 

Conclusion III 

A difference in dominant plants indicates that the Itasca and 
Gunflint forests represent two different types of forest communities. 

Forest communities are dynamic because of disturbances. Fire 
is such a disturbance. 

Some authorities say this area *has spruce-balsam climax. But 
this climax is seldom reached because of fire. 

Talk by forest ecologist M.L. Heinselman 
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SUPPORTING DATA FOR 1968 AT GUNFL I NT- INSECTS 



Quadrat 4 
TI -Culicidae- 1 

Ceratopogonidae- 1 
Dolichopidae- 1 
TI -Simulidac- 1 

Pipunculidae- 1 
Till -Simulidae-1 
Lauxanidae 3 
Pipunculidae - 1 

Quadrat 22 
TI cheronomidae- 1 
Naucoridae- 1 
TII -Eurytomidae- 1 
Till -Culicidae-1 
Phoridae-2 
Simulidae- 1 
Muscidae- 1 



rnonucic 

Quadrat 7 
Tl-no insects 
TII culicidae-1 
Tlll-no insects 



Quadrat 16 
TI Farmicoidae- 1 
TII culicidae-1 

Pipunculidae- 1 
Tin Asilidae-1 



Quadrat 10 
TI -Pipunculidae- 2 
Cheronomidae- 1 
Phoridae 
Drosophilidae- 1 
TII -Simulidae-1 

Calliphoridae- 1 
Till - Pipunculidae- 2 
Eurytomidae - 1 
Simulidae-1 



Quadrat 9 
TI -Eurytomidae- 1 
Phoridae- 1 
Dalichopidae- 1 
TI I -Lauxanidae- 1 
Phoridae- 1 
TI I r- Pipunculidae- 2 
Simulidae- 2 



Quadrat 15 

TI -Ceratopogonidae- 1 
Culididae- 1 



Quadrat 17 
TI -Culicidae-1 
Tendipedidae- 1 
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Cicadellidae- 1 
Simulidae- 1 
I chneumonidac- 1 
Sparassidae- 1 
3 unknown 

TI I - Ichneumonidae- 1 
simulidae- 3 
Ceratopogonidae- 1 
Quadrant 19 
TI -Culicidae- 1 
Phoridae- 3 
Til -Eurytomidae- 2 
Phoridae- 1 
Culicidae- 1 
Till -Lauxanidae- 2 
Eurytomidae- 1 
Culicidae- 1 
Simulidae- 2 



1 unknown 
Til- Cercopidae-1 
phoridae- 1 
Til I -Pipunculidae- 2 
Phoridae- 2 



Quadrant 24 
TI -Pipunculidae- 3 
Simulidae- 1 
TIII-Culicidae-2 

Pipunculidae- 4 
Tabanidae- 1 
Aphididae- 1 
Til -Pipunculidae- 1 
Eurytomidae- 1 
Chironomidae 
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Insect Study Results 



Diversity Index 



Gunflint 

Number o£ species 22 

Number of individ. 71 .3099 

Itasca 

Number of species 18 

Number of individ. 65 .2862 



Coefficient of community 

Number of species from Gunflint 22 
Number of species from Itasca 18 

Number of common species 4 

Gunflint 

Number of species 22 

Number of individ. 71 

Itasca 

Number of species 18 

Number of individ. 65 









Insect Study Conclusion 

Conclusion I 

The difference in the two diversity indexes indicates that the speci 
numbers relations were not quite the same in the communities. 
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Results for 1968 at Gunf lint-Soil 



Section 


potash 


nitrogen 


pH 


phosphorus 




4 


7. Vo 


0% 


3 


8% 




7 


8% 


2% 


3 


10% 




9 


8% 


0% 


2 


10% 




10 


121 


2% 


2.5 


9% 




15 


12% 


0% 


2 


8% 




17 


12% 


0% 


2 


8% 




19 


13.3% 


2% 


3.5 


10% 




24 


14% 


2.5% 


2 


9% 






Results 


for 1967 at 


Itasca 


--Soil Study 






Chemical 


Content of 


Soil and Debris Depth 


Sections 


Nitrogen 


Potash Phosphorus 


pH Debris Depth 


3 


2% 


2% 


28% 


5 1/4 


9.5 inches 


4 


2% 


2% 


26% 


4/3 


1.3 inches 


7 


3.5% 


2.3% 


26% 


5 1/4 


1.5 inches 


11 


2% 


4% 


28% 


5 1/2 


1.5 inches 


13 


2% 


4% 


28% 


5 1/4 


1.7 inches 


17 


3.5% 


2% 


28% 


5 1/4 


2 inches 


19 


2% 


8% 


20% 


3/4 


1.3 inches 


22 


2.5% 


2% 


28% 


5 1/4 


1.5 inches 



The debris was mainly composed of decaying leaves, grass, 
and wood. 



roots 



Supporting Data for 1968 at Gunflint- -Soil 
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Pattern of grid for 
ecological study of 
the Squaw Lake Island 

Scale-1 inch-100 feet 



General Information for 1967 Itasca Study 
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General Information for 1967 Itasca Study 



The data for the study was collected on an island in Squaw Lake, 
Itasca State Park, Minnesota. Squaw Lake is located in the north- 
eastern corner of the park, section S of Clearwater County. 

Papoose Island is found in the southern portion of the lake and is 
approximately 1200 feet long and 600 feet wide. An average of 501 feet 
separates the island from the east lake shore. Its distance from the 
west shore varies from 354 to 897 feet. 

The interior of the island is high ground. The east side drops 
abruptly to a marsh, forming the east shore. The water depth between 
the island and east shore is rather shallow with waterlilies growing 
there in abundance. The average depth is 29 feet. 

The west side of the island slopes gradually, the shore being 
abrupt and rocky. Off this shore the average water depth is 39 feet. 
The northern tip was the reansition zone between the marsh of the east 
shore and the west shore's abrupt drop. 

AN ECOLOGICAL STUDY OF AN ISLAND IN ITASCA STATE PARK, MINNESOTA 
III CRITERIA 

A. Planning of a project is necessary to ensure efficiency and complete 
ness and to make sure that necessary requirements-tools , location etc., 

are available. 

B. Implementation is necessary so data can be collected for analysis. 

C. Analysis is necessary to discover what the data signifies or means. 

A.l. Gunflint and Itasca sites are compared because these sites were 
available to us while working in the Mobile lab summer program. 



B 
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A. 1.1. Insects, vegetation, soils, and mammals were chosen as areas of 
study because they appeared to us to be the most significant areas in 

a community study. 

A. 2.1. A grid is used to facilitate sampling. 

A. 2. 2. Vegetation is studied because it is an important part of the 
ecological community. 

A. 2. 3. Soil is important in community. 

A. 2. 4. Insects are important in community. 

A. 2. 5. Mammals are important in the community. 

A. 2. 1.5.1* The change between 1967 and 1968 methods was necessitated 
by the fact that fewer traps were available in 1968 so sampling had 

to be cut down. 



